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ABSTRACT 

Speedie, Stuart Mitchell. Ph.D., Purdue University, 
August, 1973. A factor analytic study of specific 
cognitive abilities related to human problem solving. 
Major Professors: Donald J. Tref finger and John F. Feldhusen. 

The problem of change is one of the most urgent 

and formidable challenges faced by all individuals in 

modern society. Yet we know little of the means by 

which individuals adjust to change. The purpose of 

this study was to investigate one way by which humans 

cope with change - problem solving. More specifically, 

this study concentrated on what human abilities are 

important to efficient problem solving. First, 

however, since present: problem-solving tasks are 

insufficient samples of problem-solving behavior, 

new tasks were developed which focused oh the processes 

involved in problem solving. Thus the objectives of 

this study were as follows: (a) to develop three group 

administration tasks - simulated problem situations. 

Verbal Mazes, and concept identification tasks - that 

measured problem- solving processes; (b) to evaluate the 

reliability of these tasks; (c) to determine the 

predictability of performance on these problem- solving 

tasks from a set of human ability measures specified by 
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the Structure of Intellect model; and (d) to determine 
the underlying structure of the ability measures and 
problem- solving criteria. 

Four forms of each problem-solving task were 
developed, consisting of single and multiple solutions, 
and an alternate form of each. Two forms of each task 
were evaluated for reliability by means ^f a two week 
test-retest situation. The tasks were also evaluated 
for alternate forms reliability by simultaneous administration 
of the alternate forms of each task. 

The representative sample of human ability 
measures consisted of 17 tests from the Structure of 
Intellect model measuring each of the processes operating 
on semantic content to produce several different types 
of products. These 17 tests, along with four of the 
problem-solving criterion tasks, were administered 
to a sample of U90 fifth grade students over a period 
of five hours. The resulting data was subjected to 
regression analysis, conventional factor analysis with 
Promax oblique rotation, canonical correlation analysis, 
and extension loadings analysis, 

Test-retest reliabilities were found to be quite 
low, ranging from .30 to .74. Alternate forms reliabilities 
were essentially zero except for the Verbal Maze problems. 
Regression analysis revealed that convergent production 
of semantic transformations and two memory abilities were 
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related to performance on the simulated problems; memory 
and evaluation abilities were related to Verbal Maze 
performance; and logical reasoning and noticing details 
were important to concept identification. The factor 
analysis revealed five factors: (a) a general test 
performance factor, (b) a simulated problem factor, 
(c) a concept identification factor, (d) a divergent 
thinking measure, and one insignificant crossover factor. 
Canonical correlation analysis and extension loading 
analysis added nothing more. 

It was concluded that, although a number of actors 
may have affected the results, the most probable cause 
of the lack of important results was the low reliabilities 
of the problem- solving criteria. The regression analyses 
yielded some information, but hardly of great significance, 
since multiple Rs were generally low. Thus the study 
requires replication with several important changes in 
order to reveal significant insights into the problem- 
solving process. Also it was concluded that no empirically 
based model of instruction could be based on 
the results of this study. -'^ 
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BEST CdH MMUBIE 

INTRODUCTION 

The problem of change is one of the most urgent 
and formidable challenges faced by all individuals 
in modern nociety. Yet we knov; little of the means 
by which individuals adjust to change, especially 
rapid change. In a world where technology has made 
change a constant faet of life, it is necessary to 
find ways to educate individuals in coping effectively, 
or face the alternative of being drowned by the rising 
tides of our own inventiveness and technology. In 
his book Future Shock » Toffler (1970) contends tnat we 
shall need individuals 

... who can make critical judgements, 
who can weave their' way through novel 
environments, who are quick to spot new 
relationships in the rapidly changing 
reality. It (modern society) requires 
men who in C. P. Snow's compelling • 
term, have the future in their bones, 
(p. 1*03).* 

Since solving the problems generated by change 
•appears to be of prime importance for surviving in 
this prescntday world, a central role of education 
should be to assist each individual in developing the 
abilities and skills necessary for coping with this 
change. Thus, schools should have the development 
of problem-solving skills in each person as one of 
their primary goals ♦ Or as Toffler (19 70) argues: 
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The new education must teach 
the individual how to classify and 
reclassify information, how to evaluate 
its veiacity, how to change categories 
when necessary, how to move from the . 
concrete to the abstract and back, 
how to look at problems from a new 
direction— how to teach him'self. 
(p. ^1^). 

' . Such' goals are often stated in the general philosophy 

and objectives of school systems but not generally 
implemented to the fullest possible extent* Students 
are of ten taught- known solutions to familiar or age- 
old problems, and little time is spent in explicitly 
teaching strategies for encountering and dealing with 
new problems. (cf. K020I, 1967; Holt, 1964). 

It is not reasonable, however, to blame the 
schools entirely for thia deficit since social and 
behavioral scientists know so little themselves about 
how people solve the type cf problems one encounters in 
the fastpaced world of today. What seems to be called 
for is the development of innovative approaches to 
instruction aimed at training individuals to deal with 
such problems. Such approaches would necessarily 
concentrate on teaching complex cognitive skills 
and strategies necessary for confronting the problems 
of fast-paced change. 

However, the effectiveness of many innovative 
I instructional programs is open to question. This has 
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been documented by Charles Silberman in Crisis in the 
Classroom ( 1970) : 

... the 19 50 ^s and ^5 0s saw one of the 
largest and most sustained educational 
reform movements in American history, 
an effort that many observers, this writer 
included, thought would transform the 
schools. Nothing of the sort has 
happened; the reform movement has 
produced innumerable changes, and yet 
the schools themselves are largely 
unchanged. (p. 158). 

Perhaps one of the reasons that few educational 
innovations have been effective is that these programs 
are seldom derived from empirically tested principles 
of instruction, nor based on proven ideas concerning 
their subject matter. Most are quite effective in 
training students to achieve knowledge level objectives 
but few have concentrated on the processes o r strategies 
important to their particular concerns. 

To summarize, our rapidly changing society requires 
persons who are able to solve a great variety of 
challenging problems. This implies that schools must 
emphasize courses of instruction which dwell on the 
skills and abilities necessary for problem solving. 
Yet, few such programs of instruction exist. Thus, 
a new and innovative approach would seem, to be 
indicated although educational innovations have a 
history of failure. Kow, then, can we proceed to 
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design innovative courses of instruction which. -will 
help our children to meet future challenges'? 

This project assumes that a number of goals must 
be accomplished in order to design such a program. 
■First, a clear and precise definition of problem- 
solving must be developed. Second, the strategies 
which people solve the multitude of problems they face 
must be elucidated. Third, the intellectual skills 
and abilities necessary for solving problems must be 
established. Fourth, instructional strategies must 
be designed which are derived from the findings 
of the first three goals and the empirically established 
principles of learning and instruction. 

Obviously, the accomplishmentof these four goals 
is -a highly ambitious undertaking and one which is 
beyond the scope of this dissertation. Instead, two 
goals will occupy the primary focus. That is, which 
intellectual skills and abilities are important to 
efficient problem-solving and can an instructional 
program based on these abilities be designed? 
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REVIEW OF LITERATURE 



The following sections will review the literature 
with respect to definitions of problem-solving. Several 
models of the problems-solving process will be reviewed. 
The operational definitions of problem-solving will . 
be discussed and evaluated. From the components of 
the problem-solving models and the literature 
concerned with intellectual skills involved in problem- 
solving, a set of important skills will be hypothesized 
and a study proposed. 

Definitions of problem-solving. 

By its semantic nature, problem solving is an 
action performed upon an object. Thus in defining 
problem solving, one must first deal with the object, 
i.e.^the nature of "problem" must be explained. Johhson 
(1972j p* 25 > stated that a' problem exists when an 
individual is motivated toward a goal and his first 
attempt to reach it is unrewarding.' Newell and Simon 
(197 2) contend that a person is confronted with a 
problem when he wants something and does not know 
immediately what specific actions he can perform to 
get it. It appears to be a generally accepted 
definition (cf. Duncker, 19H5; Wertheimer, 1959; 
Duncan, 1959; Ausubel and Robinson, 1969; Bourne, 
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Ekstrand and Dominowski, (1971) that a problem is a 
situation in which a person is motivated toward a goal 
froni some present state,- but does not know the specific 
means of achieving that goal, given the present state. 

Problem solving is, then, the means by which an ' 
individual moves from his present state to some goal. 
However, what exactly constitutes this process is a 
matter for considerable debate. Davis (1966) claimed 
that problem solving is any semi-complex learning task 
which has not already been identified by another name. 
The more concrete definitions, however, have been 
expressed as identifiable subgroups of intellectual 
processes wh-ich are linked in some order. These 
links have been of two kinds: linear. and feedback. 

Earlier definitions of problem solving generally 
listed several intellectual processes linked in a 
linear manner. Dewey (1933) proposed that problem 
solving consisted of five phases: 

foT^w^^H^^^'^^^^-i?''^ "-"^ ^^^^ leaps 
forward to possible solution; (2) an in- 

tellectualization of the difficulty or 

fo''So^''^*^ ^^^"^ ^^1^ i^to^ problem 

lnst\l°ll^^l.^ question ?5?-which the answer 
must be sought; (3) the use of one surrestion 
to Ini??aS''' f a leading idea, or hf^o?Sis?s, 
to initiate and guide observation and other 
operations in the collection of factual 
^H.ri^^^' (^).the mental elaboration of the 
ivnni-?L-=^r^'-^'-°'' ^'^^ testing the 

(p 107J imaginative action. 
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Wallas (1S26) suggested four phases of problem solving 
which were preparation, incubation, illumination, and 
verification. Later definitions of problem solving 
suggested interactions among the various phases. 
Merrifield, Guilford, Christensen and Frick (1952) 
proposed that the phases of preparation, analysis, 
production, verification, and reapplication interacted 
in the problem solving process. Johnson (1972) 
suggested three broad classes of processes: preparation, 
production, and judgement. Others, such as Gagnd (1965) 
maintain that problem solving is the complex interaction 
of a number of subordinate learnings which load to the 
learning of a new rule . 

Later approaches to defining problem solving have 
usually considered it to be the operation of an in- 
formation processing system, a system of processing 
and feedback. A paradigm of such an information 
processing system is given in Figure 1.- 
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Figure 1. General structure of an information processing 
system (After Newell and Simon, 1972). 



Newell and Simon (1972) make four propositions 
concerning problem solving in this information processing 
system: 



A fev;, and only ^a feWj gross characteristics 
of the human IPS are invariant over task and 
problem solver. 

These characteristics are ' sufficient to 
determine that a task environment is represented 
(in the IPS) as a problem space, and that 
problem-solving takes place in a problem 
space. 

The structure of the task environment 
determines the possible structures of the 
problem space. 

The structure of the problem - space determines 
the possible programs that can be used for 
problem-solving • 
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terms, that: (a) there are only a fev? characteristics 
which are true of all problems and problem solvers; 

(b) but these few characteristics are enough to determine 
that the problem. is perceived by the problem solver; 

(c) and these objective characteristics of the problem 
task determine how the problem solver views the problem'i 

(d) and finally, how the problem solver perceives 
the problem determines how he will solve the problem. 
Newell and Simon have derived this model from a great 
deal of simulation v/orJc with three problem tasks : 
theorem proving, cryptarithmetic^, and chess. For 

these tasks, possible problem spaces have been specified, 
and programs which appear to imitate human behavior have 
been developed* However, one must be cautioned^ that 
these problem spaces and programs (solution strategies) 
are highly task specific. 

Guilford (1967) proposed a model of problem solving 
involving processes from the Structure of Intellect 
model. This model is represented in Figure 2. In this 
model the problem exists in terms of environmental and 
somatic input. This input is filtered by arousal and 
attentional mechanisms and the problem is either ignored 
(Exit I) or the problem is sensed and structured 
through cognition processes. These cognition processes 
may call on evaluation processes or new input. After 
passing through this stage, the problem may again be 
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exited (Exit II) or answers generated either. by 
convergent or divergent production. These answers are 
evaluated and either the problem is exited (Exit III) 
or a nev: cycle of cosnition, production, evaluation 
is started. Underlying all of these processes is 
jnemory accession^ upon which the other . processes are 
dependent. The Guilford inodel of problem solving is 
an information processing model in which a variety of 
intellectual processes act upon environmental and 
somatic input , vzith constant reference to memory 5 to 
generate problem solution^n. 

A third m.odel of problem solving is suggested by 
AuGubel and Robinson (196?). The models, as depicted 
in Figure 3, postulates four levels in the problem.- 
solving process: (i) problem setting, (2) definition 
of the problem, (3) gap filling, and (4) verification o 
the solution. Central to problem solving is. the gap 
filling process* This process starts with the given 
problem information plus an individual's background. 
These are manipulated by rules of inference guided by a 
strategy in order to reduce the 'gap between the initial 
proposition and the final proposition . or goal. This 
gap filling process is similar in many respects to the 
means-ends analysis of Newell, Shaw and Simon (1958) 
which they employed in their General Problem Solver. 

' 22 
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A fourth model of problem solving was developed by 
Feldhusen, Koutz, and Ringenbach (1972) through a review 
and synthesis of the problem-solving literature. As a 
result of their review, they postulated! 12 different 
types of problem-solving processes: 



1. 


Sensing that a problem e>:ists; 


2. 


Defining the problem; 


3. 


Clarifying the problem; 


4. 


Asking questions : 


5. 


Guessing causes; 


5. 


Judging if more information is needed; 

■Ik 


7. 


Noticing relevant details; 


8. 


Using familiar objects in unfamiliar wa2/s ; 


0 


Seeing implications ; 


•10. 


Solving multiple solution problems; 


11- 


Solving single solution problems; and 


1?. 


Verifying solutions . 



A fifth, perhaps more elemental model of problem 
•solving was put forth by Miller, Galanter, and Pribram 
in their book Plans and the Structure of Behavior (1960). 
They postulated that ^all variety of problems confronting 
human beings are solved through the use of ''plans *^ The 
basic element of the ''plan'' is the TOTE. The TOTE 
is a basic feedback system illustrated in Figure 
The basic process is to test a given condition, exit 
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if the condition is satisfied, otherwise perforjn some • 
one of a sm^ll class of . 'operators and test the 
condition again. It is evident that this model does not 
specify any particular processes involved in problem 



Input 



TEST 



(Incongruity) 




Output 



j (Congruity) 



Figure 4. The basic TOTE feedback system. 



solving, but it does specify the form that any such 
process will take; namely a TOTE hierarchy as given 
Figure 5. 
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Figure 5. A TOTi; hierarchy for hammering a nail. 

To summarize 5 there are a number of definitions of 
problem solving, embodied in several different models. 
These definitions all specify that problem-solving is 
the process of reaching a goal state from an initial 
state, where the means is not at first known. However^ 
the intellectual skills and abilities that choose, 
initiate-, and execute these processes is a matter of 
debate. It remains a matter for empirical study to- 
identify these intellectual skills and abilities. 
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Esipirical studies of a -bilities .-,-mDor>-t.-,.nt- to ^r^oblem 

•Basic empirical work in this area was done by 
Merrifield et al. (1962) in their factor analytic 
study of problem solving. Further analysis of the 
data collected were presented in Guilford and Hoepfner 
(1971).- In this study 37 measures representing 16 
different Structure of Intellect (SI) "factors were 
administered'^ to a group, of 232 Naval personnel. The 
16 SI factors represented are listed below: 

1. cm cognition of semantic units (verbal 

comprehension) 

2. CMC cognition of semantic classes (con- 

ceptual classification) 

3. CMR cognition of semantic relations 

(education of conceptual relations) ■ 
^. CMS cognition of semantic systems (general 
reasoning) 

5. CMT cognition of semantic transformations 

(penetration) 

6. CMI cognition of semantic implications 

(conceptual foresightT > 

7. DMU divergent production of semantic units 

(ideational fluency) 

8. DMC divergent production of semantic classe 

•• (spontaneous flexibility) 
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9. DMR divergent production of semantic 

relations Cassociational fluency) 

10. DMT divergent production of semantic 

transformations (originality) 

11. NMC convergent .production of semantic 

classes 

12. NMR convergent production of semantic 

.relations (education of conceptual 
correlates) 

13. NMT convergent production of seitiantic 

transformations (conceptual redefinition) 

14 . NMI convergent production of semantic 

implications (deduction) 

15. CMR evaluation of semantic relations 

(logical evaluation) 

16. EMI evaluation of semantic implications 

(sensitivity to problems) 

central reason for choosing these particular 16 
the authors reasoned hypothesis that these 
litieswere central to the production phase of problem 
ving. They had chosen to concentrate on this phase 
to the nature of their problem-solving tasks. 
The problem-solving criteria of the study were 
sing Links » in which the task was to produce three 
ds to complete a- chain of associations between an 
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initial and final uorff. , .The .econd criteria ..s 

SSSiiSSmrts in „hich the' task was to produce two 

ways in which given objects c^uxd be used to solve a ' 

specific problem. A third criteria was 

in which the S was reauir'-d tn --^.-^ 

--qair^a lo -^rite a coiierent account 

logically connecting the given in^'ti;,! .r,^ • 

c-vtiii initial and fxnal sit- 

uations Of a short story; this test yielded measures . 

Of coherence and logical connection. A summary of . 
their revised results is given in Table 1. 

The ...ost recent analysis (Guilford and Hoepfner, 
1971) indicated that there was no discrete problen.- 
HOlving factor. Rather, the problem-solving tests 
loaded on a nu^.ber of SI factors. Transitions was 
associated with an ability they lab^^I^T^l 

comprehension Cor. in ■>-^>^^c^ • . 

^o.. m bi cerras, cognition of semantic • 

units). Performance on the Predicame^ test was 
related to performance on a factor they labelled 
conceptual foresight or cognition of sen^antic i:n- 
plications. The ability labelled sensitivity to 
problems or cognition of s^ntic implications was 

related to performance on both the Pr,o^-:<, 

L>ju.i xne ir;redicaments and 

lESHlitions tests. W^._lAnl<s_,^, associated with 
convergent production of semantic transformations and 
finally, originality or divergent production of semantic 
transformations and performance on the IWitions test 
were shown to be associated. 
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However, it should not be concluded from the 
report of Merrifield et. al. (1962) that these are the 
only abilities related to human problem solving. One 
serious limitation of the study involves the represen- 
tativeness of the Criterion tests with respect to 
problem- solving behavior. Three of the four measures 
were based on the determination of verbal associations, 
utilizing simple word to word associations in the 
Missing Links test or the more complex "implied meaning" 

Transitions test. In the Predicam.ents test, 
the dominant ability would appear to be the recall 
and evaluation of uses for the objects specified. In 
none of these tests was there a provision for measuring 
the strategies of problem solution, or provision for 
m.ultiple solutions, asking relevant questions about 
the problem, clarifying the goal, or defining the 
problem. Thus, the tests em.ployed by Merrifield et. al. 
(1962) did not include measures for many possible facets 
of efficient problem solving. 

Werdelin (1956) reported two factor analyses in 
which the criteria were problem-solving tests in 
mathematics. In one study 36 human ability tests were 
administered to a group of boys, and in the second study 
29 tests were given to similar samples. There were 18 
tests in common in the two studies. Factor analysis 
resulted in 5 common factors for the human abilities. 
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They were general reasoning, deductive reasoning, 
verbal comprehension, space and nu:nerical factors. The 
problem-solving criteria loaded n^ost heavily on the 
general reasoning factor. A.,-ain, however, the .en- 
eralizability of the factors was limited by the specific 
nature of the criterion rests, 

A third factor analytic study, carried out by 
Sanderson (1957), used probleir-solving criteria of 
• concept learning tasks. The pri^r.ary purpose of the 
study was to determine the relative importance of a 
number of hu.-.an abilities at several stages of concept ' 
learning. A total of 30 tests' representing 10 factors 
and 26 concept problems were ad:r:inistered to a sair.ple ' 
of 1U5 Princeton undergraduates. For complex positive 
and negative problems, the concept problems were 
divided into 5 sequential blocks of 3 problems each and 
three measures were derived. The remaining eight 
problems (double negative) were divided into 4 sequential 
blocks of 2 problems each. Extension loadings of the 
concept measures on the ten factors were calculated 
using a differential performance function, for each of 
the blocks, 

Bunderson reported that factors of Chunking Memory, 
Perceptual Speed, and. Spatial Scanning , were important 
in the initial attempts at solving the problem.s. The 
factors of Verbal and General Reasoning, Induction, 
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and Flexibility came to play a more important function 
as Ss iir.prbved in solving concept problems. And finally 
the factors of Memory Span, AssociativFTleinory , and 
Spatial Scanning became most important in the latter 
stages of the concept problems , presumably when 
strategies for solution had been learnedj» recall 
became the principle ir.eans of solution. Again 3 however, 
generalizability of the results to other problem- 
solving tasks is limited by the specificity of the. 
criteria. 

These three studies comprise the significant 
literature in the factor analytic studies of problem 
solving. However, two other studies have used a 
correlational approach to the investigation of what 
human abilities are important to problem solving. 
Haro'otunian and Tate (195 0) report a study in which they 
attempt to relate a com.posite problem-solving criterion 
to a number of human ability factors through regression 
analysis. The factors included Problem Recognition, 
Word riuency. Ideational Fluency, Closure, and Judgement. 
These were represented by a total of 14 tests. The 
problem-solvin?^ criterion consisted of a linear 
composite of verbal and abstract reasoning scores from 
the Differential Aptitude Te'st , the Davis-Eells Games , 
and a specially constructed test "entitled Thought 
Problems . The battery of tests was administered to a 
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group 632 seventh and eighth grade students • Factor ■ 
scores were additive composites of their representative 
tests. Correlations of the factors with the problem 
solving criterion were respectively •62U^ .^l?, ,290^ 
.395, and .707/ In the light of the nature of the 
criterion, these correlations are not surprising. 
There is a high degree of similarity between the tests 
representing Problem Recognition and Judgement and the * 
three tests making up the criterion. Thus, this study 
does not provide much information concerning the important 
factors in problem solving, though it does make an 
attempt to use more a comprehensive measure of problem 
solving. 

The second correlational study was by Stevenson > 
Hale, Klein, and Miller (1953). In this study the 
aut}.Yors investigated the relationships between a num.ber 
of learning measures and several problem-solving tasks. 
The learning measures consisted of 12 tasks, tv;o paired 
associate tasks, three discrimination tasks, a 
probability learning task, an incidental learning^.task, 
and a verbal memory task. ■ The problem-solving criteria 
were two concept of probability learning tasks, a 
conservation of volume task, and an anagram task. 
These tasks were administered to a group of bright^ 
average, and dull seventh graders. 
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It was evident froir.- their --lata that there were a number 
cf sex differences. For both sexes, however, the corr- 
■ elations show that associative memory as reflected in 
•the paired associate tasks is important to performance 
on the problem-solving tasks. This conclusion is rein-, 
forced by the importance of verbal memory in the 
problem-solving tasks for girls- The authors conclude 
that 

"The high proportion of significant 
correlations across learning and problem- 
solving tasks reveals the comjnon dependence 
of the tv7o categories of tasks upon^Ss' 
ability to acquire new information and to apply 
this information on subsequent trials and in 
the solution of new problems." (p. !+7) ■ 

There is considerable difficulty in drawing any 
conclusions from these studies with respect to human 
abilities important to problem-solving. The primary 
reason for this is the lack of similarity among the 
problem-solving criteria used in . the different studies. 
The most that can be said is that some verbal reasoning 
^nd memory abilities appear consistently across studies. 
In order to establish firmly what abilities are important 
to problem solving, it is necessary to have comprehensive 
and representative m.easurcs of human, problem.-solving which 
are operational definitions of the definition established 
earlier in this paper. 
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Ch aracteristics of Id eal Problem r^olvn^n^jr^ 



Cronbach (1955) was concerned with the testing 
situation which would elicit the maximum response. 
Thu-^hG proposed that problems must be meaningful 
to the subject, must not degenerate into exercises" 
which make no demands on the higher m.ental processes, 
and that the problem setting should be considered 
carefully to rem.ove distracting factors.. Ray (1955) 
suggested two additional requirements for problem- 
solving tests; that they should have a scoring continuum 
rather than a simple pass-fail. score?' and that they 
should provide the test administrator with the maximum 
amount of knowledge about what the subject is doing. 
John (1957) put forth a number of criteria for problem- 
solving^ tests in accordance with his dictum that the 
prcblem-solving process should be accessible for direct 
observation.- He proposed that such tests should (1) 
start the subject with a standard minimum of information 
about a problem and then require him to structure his 
own presolution behavior with a minimum of externally 
imposed constraints i (2) be maximally free of special 
skills, special knowledge, or experiences peculiar to a 
given culture 5 (3) be constructed so that the effect 
of familiarity with the generic tasks is minimal in 
order to facilitate the construction of equivalent forms; 
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and (4) have a format which presents the subject with the necessity 
to interact with real events rather than abstract. 

These criteria have been principally conccsrned with 

i 

problem solving in a laboratory setting. Koislcr (1969) 
took a different point of vioW; in that he was interostecl in 
tests ifsablo in the classroom. His criteria are summarized 
in the foi lowing I I points: 

1. The population of problems sampled by the test 
, - ^ - ■ . w should be appropriately defined. 

2. The population of problems should be an important 
one. 

3. The test should not be inadvertently confounded 
with assessment of prerequistie learnings. 

4. The conditions under which the test is given must 
be clearly specified. 

5. The test should indicate how well the child was 
following the appropriate procedure leading to 
the solution. 

6. Increasing number of cues should be used to 

find out how much assistance the learner requires. 

7. On some occasions the student should be required 
to provide an oral or written account of his own 
covert procedures, assuming such self-report 
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does not i n 1 1- r f e r c vj i t ;i t h e p ro i. 1 em- s o 1 v in;^ 
proaestj * 

vaalit'/ of proMeir oolvinr can ::e xvr^essed 
turcugn tiie recxirvic of piw'j*-:urcs ana «io-.utions. 
J. rroliein betting, at disiin^; f : cir : prcii^lem- 

oolvinr., s:.o-. Ic lio'. ::e aj.:.e.3ru uncitir standard- 
ised condl t J.orii> . 

also we con:^! :.ere.u . 
11* At a practical level, a tesz of proMeiT: solving 
rUiOUlc Le araonaDle to, group presentation if at 
all po::>3i:>lt; , 

Covinf^ton (ir. prr-^sis) reflected t:iese concerns by 
concluding that teciifiiques are needed whioii rcfTect, 
as fully a3 poci^it'le, the ricii complexitieG of creative 
tnought, Wiiici; allO'A for t];e distinctive dispositions and 
cognitive styles of tne?* creative person, Lu' v/nich, at the 
same time, permit a reasonable degree of standardization 
and the uco of objective, scoring procedures. 

In terms of concerete criteria for probleir.-solving 
tests ail of the uointf^ made by the abcv^* writers appear 
to be summarized in tne fdllovjinp, : 

1,, .Tasks selectea for protlem-^nolving tests should be 
complex; i,e., they shoulu not bo merely si.xple 
exercises, out ratiier problems in which 
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Tnerc are . a large nuniber' of steps from en 
Initial st-.t? to ^. final state, cr a reasonab 
large number of attributes. • 

2. Perfcrr.anc^' on the test c>culd h^. minimally 
related to previous learning which could 
differentiate individuals at the tijne of the 
test. 

3. The problems should coir.inand the attenti-cn and 
interest of the subject so as to insure r?.n 
adequate level of motivation for optinum 

per f ormanc e , 

•4. The test should yield a variety of continuous 
measures concerning the outcomes of problem 
solving > the processes, and the intellectual 
s'^:ill3 involved. 

5. The test should contain a m.inimum nur.ber of 
constrain-cs on th^ type??, of problem-solving 
behavior the individval may enga^^e in. 

6. ■ The test should demonstrate both reliability 

and validity. 

7. The test should be practical for group 
administration . 

Problem Solvin jg ; Tests 

A number of tests have been developed to measure 
problem, solving in general or in specific areas, but it 
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i.3 debatable t--hether or net any meet the above criter: 
for an ideal test. Feldhusen et. al. (1971) report 
four classes of tasks present in the literature which 
have been used to investigate problem solvin' 
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ar-B ■;i) puzzle-insight proolerns, C2) process oroblerns, 
(3) cor.porent tasks, and real-life, relevant tasks. 

The fii-st sot of tasks, the puzzle-insight problems, 
have been the tools for :?.cst of the classical investi- 
gations of probler. solvin.^. Perhaps the r.ost far.ous of 
those are Maier's T'-To-Stri-.q; and Hatrack (13U5) problems, 
D-o.ncker's (IS^^S) box proble-., 'Luchin's (1942) Water Jar 
pr-blens, Katona's (18i;0) Katchsticks probleriis and anagrams 
(Johnson, 12o6). All these tasks are basically artificial 
intellectual garr.es in v;h5.ch txie initial conditions and 
final goal are precisely stated, and there are a 
severely limited number of "'..'ays to go between tiie •^'^o 

The second type of problc.Ti task, the process 
problems, have been used in research which focuses 
prr.narily on the processes of prcblen solving. This 
type includes switchli.'j.ht problems (John, 1957 ; Tyler, 
1955; Davis, 1956; Davis, Xanske, and Train, IB&S), 
the verbal maze problens of Hayes (1965), simulated 
problem situations, (Glaser, Damrin, and Gardner, 
195--!; RiToldi, 1960; HcQuire and Babbott, 1967 ; Strcufert, 
Kli/'jer, Castore, and Driver, 1967: Mattress, 1971), and 
concept identification (Bruner, Goodnow and Austin, 195S; 

Ik 

« 

39 



Clark, 1971', Bourne . -Lkstrand , S Dominowski, 1971). i'he 
comTTion element to each of these tasks is a structure 
Vwhere there are a nur.ber of discrete decision points 
where an individual ^5 choices can be recorded. The 
rerult is a record of all a Gubjecc's decisions throughout 
thf; course of the problem vrhich can be st,udied and pro- 
ce:;se5 and strategies inferred. * 

The corponcnt skills and abilities involved in 
problem solving are the primary concern of t;ie type ^ 
labelled ''conponent problems.'' Included in this category 
are the ..-attery of tests used by Guilford (13G7; to ^ 
establish the Structure cf Intellect model of intelligence, 
the Torrance Tests of Creative Thinkinc; (TTCT; l:o5a), 
Unfinished Stories (Lundstoen and Michael, 1S66). and the 
Purdue -Slementary Trobleri ?olvin <:; Inventory ( ?e Id hu sen 
et. al., 1971). This type of test is based on a 
the'^retical approach V7hich maintains that problem solving 
is carried out by the efficient employment of a /:ide 
variety of skills. Thus in order to measure problem 
solvinr behavior, it is necessary to observe behavior 
related to each one of these skills. Consequently, each 
of the tests is designed to tap one specific ability by 
presenting highly specific problem situations which 
precisely defined initial conditions and criteria. for 
problem solutions. 
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The fourth type of problem solving test ccnsxsts 
of those tests which placo great emphasis on problem 
situations which :?epresent "real-life" situations 
and c.re relevant to the concerns of the student. - Tests 
of this typ-2 are Crutchxield and ::ovington's (195?) 
Ij?g.,P ld Black H ouse, Thejfa n in the Pit , and The Missing 

Miles (19 68 ) Creative Desi.? n Te^. and Treffinger's 
(1^70) Solving Probl emsJ_2. These are tests similar to 
the component type in that the measures are the sar^e 
but they place greater emphasis on T.otivating Ss to 
perform as -/■ell as possible by solving relevant problems. 

Ev-zluation of Problem Solvi ng Tes t s 

The instruments from these four categories of 
problem-solving naasureSv however , do not- nc,cessarily 
■rce.e-^'all of the criteria for an ideal problem-solving 
test. V/ith respect to the -.criterion of problem 
complexity the pu"?,le-insight problems do not qualify, 
since the tasks are fairly simple. in nature, having onlv 
a few components to work with. The other three types do 
meet this criterion in that for any of these problems 
the structure may be as complex as desired. As an 
exampl-e consider the medical diagnosis sir.ulation. When 
the S is first given the problem he has a number of choices 
as to action and contingent upon this choice is additional 
information and other choices which lead throuo-h a 
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large network of decisions to a final diagnosis and 
treatment. 

The second criterion, that of effect of previous 
learning, i.-= particularly important to those types of 
tests which are hijhly task specific in nature. Kost 
of the puzzle-insight problems have a relatplvely simple 
if unusual solution or strategy for solution which is 
easily re.-nerr.bered once it has been encountered. Thus 
for those people who have encountered the task before it 
beccr.ec a test of meniory rather than a test of problem 
solving. This would appear to be a difficulty with ell 
those which strive for relevancy, since if the problem 
situation resembles too closely past experience, r.emory 
becomes the do:?inant influence rather than more complex, 
and perhaps more important, skills. Those tasks which 
•depend little on past experoince 'or on a level of 
experience common to all Ss . such as concept identification, 
verbal mazes , some of Guilford's tests, or the TTCT, 
should be more able to tap skills other than memory. • 

Unfortunately, those same tests which are the least 
affected by past experience are tiiose most likely fail 
the third criterion, an adequate level of motivation. 
The prime purpose of this criteri-on is to insure a 
generally uniform level of motivation to undertake the 
problen task. When the task cannot be related to past 
experience, <fther types of motivating forces, such as 
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intellectual curiosity, social pre^.sure* or exnerimenter 
pressure nu3t be relieri upon to produce the motivating 
force. Yet these vary greatly from person to person 
in terms of susceptibility so that a much larger range of 
motivation is produced and there is consequently a 
s|»uriously increased range of performance on the task. 
Thus these tests which are least likely to hold the 
interest of the student such as the r.atchstick problems, 
concept identification problems, or the TTCT are most 
likely to fail to meet this criterion, while the medical 
simulation, the v;ar ^ame simulation, the FSP3I, and 
The Old Black IJouse arc more likely fc capture the 
interest of the student and thus provide more uniform 
levels of motivation. 

The fourth criterion. v;hich is concerned with the 
neas.urcs provided by the test, is particular to the 
puzzle-insight problem.s. In order for a problem- 
solving test to be of maximum utility, it must yield a 
TTiaximum amount of information about the Droblem.-solving 
behavior of the student. The pu2::>:le-insight problems 
have been primarily concerned v;ith t5.me to solution, 
number of mistakes or inappropriate solutions, and 
number of hints necessary for solution. These are 
strictl>' end-product measures and provide little 
information about the processes or skills involved in 
problem, solving. The process problems supply a great 
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deal more information , due to the fact that the S 

is follov/ed through the problem by the decisions 

he makes at each point. Thus in addition to the end- 

proGuct measures, information is also available on the 

strategies used, the efficiency of the strategies, and 

the sequence of typesof decisions. The component and 

the 'Yeal-iife^- problems provide a different kind of 

information. Again these are end product measures^ 

but these have to do with the proficiency of the 

individual in the individual skills which theoretically 

facilitate problem solvinj behavior. Thus it would 

app(!jar that the process tasks yield the greatest 

amount of information about actual problem.-solving bejiavior 

and consequently best satisfy the fourth criterion. 

Constraints, the subject of the fifth criterion ^ 
are evident in all of the tasks cited above. The only 
issue is which one demonstrates the least number of 
constraints, since it is obvious that some constraints 
are necessary, in order that problem solving behavior can 
•occur at all. However, some types of problems such as 
the puzzle-insight type present a high number of 
constraints upon the strategies usable for solution or 
the solution itself. The component problems generally 
suffer from the sam.e difficulty due to the highly 
specific nature of the tasks. Perhaps the process 
problems again best satisfy this criterion since they 
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provide a number cf possible paths to solution, though 

the multiple solution component problems such as the 

TTCT also have, a low number of constraints on the solution. 

The reliability and validity of problem-solving 
tests, the sixth criterion, has not been a subject for 
intense investigation. In .-lost research in problem 
solving, the task has been the object of investigation 
rather than the task as a manifestation of a larger 
and more generalised set of behaviors. In such cases 
reliability was unimportant since the individual only 
performed the task once, and validity was irrelevant 
since generalization to other types of problems was 
not attempted. However, some validity evidence does 
exist. Mendelson, Griswold and Anderson (1955 )^reported 
that performance on anagrams was related to a measure 
of intelligence. A cSnsistent positive relationship 
between intelligence and concept identification per- 
form.ance has been demonstrated Cl-Ioffman , 1955 ; Osier 
and Fivel, 1951 j Denny, 196B; Vrnitman, 1966 ; Bunderson, 
1967). The validity of the TTCT has been investigated 
and found acceptable in a number of studies (Torrance, 
1966b). But, for the majority of existing probler. 
solving tasks , the criterion is not satisfied. 

The final criterion, practicality for classroom 
administration, is satisfied by few of the existincT 
measures. Most of the puzzle-insight problems are 
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indjA'idually administered as are the verbal nazss, 
Gwitchlight problems, and concept identification tasks. 
The Katchstick problems (Katona, iguQ), anagrams, simulated 
probler?. situo.tionr> and all of the component and "real- 
life'" problems exr.r.t in a form for group administration. 
Kovevsr scoring is often a difficulty with these tests, 
since responses to such tests as the TTCT, and the 
Creativ e Des ign^Test, must be rated by judges in order 
to derive num.erical scores. Thus it would appear that 
most of the tests are beset v;ith problems in relation 
to practical concerns. 

In choosing a representative set of tests of problem- 
solving behavior according to the stated criteria, it 
xvould appear that those measures in the process category 
are best qualified as ideal problem.-solving tests. 
That is, they should yield the maximum am.ount of in- 
form.ation about the ,problem-solving process and reflect 
the utilization of a sizeable number of hum.an abilities 
and skills. The component problems-, while they appear 
to qualify by m.ost of the criteria , are not useable 
due' to the fact that they are based on pre-existing 
theories concerning the skills involved in problem- ' 
solving behavior. 

Yet as stated above , the tests in the process 
catsrory are far from perfect. A number of improvements 
need to be made before they would come close to meeting 
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the criteria for "an idaal test. The verbal maze problems, 
ne-d a higher level of motivatio.n , evidence for 
reliability and validity of the measures yielded, and a 
forn vhich i. a..e.->able to a large .rouf testing situation: 
The simulated problem situations satisfy most of the 
criteria except that they need to be constructed so as 
to rely less on past experience, and be supported by 
more evidence for their reliability and validity. The 
concept identification tasxs need t higher level of 
motivation, and a. form which is useable in classroom 
testing. Finally all of the tests are restricted to single 
solution problems while for purposes of complexity and 
verisimilitude multiple solution problems would be 
appropriate . 

Thus it would appear that a set of tests which would 
provide the most representative sample of problem solving ' 
behavior are the process problem.s , specifically m.od- 
ifications of the verbal mazes, the simulated problem 
situations, and the concept identification tasks. 

Summ.ary . 

Problem-solving has been defined as the process 
by which the problem solver starts from an initial 
state and proceeds to a goal, where the specific means 
by which this is accomplished are not initially known 
to the problem solver. The models of problem solving 
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which x-jeve discussed, hypothesized that a na-nber of 
different processes and hu-nan abilities were involved 
in problem solving, i.e. it is a complex cognitive skill, 
The crnpirical studies of haman problen-solving abilities 
have revealed that this hypothetical assertion has some 
basis in reality. Finally a wmhev of criteria . for a 
sufficient operational definition of problem solving 
were discussed, and it was concluded that tests which 
are concerned with processes in problem situations met 
. most of these criteria. 

Problem Statement 

The purpose of this study is to establish a set 
of human abilities v;hich are related to efficient 
human problem solving. Specifically, this study will 
build upon the results of Kerrif ield , et al. (1962) 
by .attempting to relate a number of hum.an abilities as 
specified by the SI model of Guilford (1967) to a set 

J'^^^^^'^''^'''-^S, measures which meet the criteria for 
an ideal problem-Solving test. These SI tests will 
be concerned with all the operations applied to semantic 
content that result in products identified as relations, 
transformations, and implications, through which in 
certain cases may include units, classes and systems. 
The only content dealt with is semantic since the tests 
must necessarily present problem situations through 
verbal statements and responses must be made in the same 
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manner. Al?. operations are included since it is possible 
to reasonably identify actions taken in problem solution 
which might involve cognition, memory, divergent and 
ccr.vercent production, and evaluation. Also the products 
emphasized are relations, systerr.s , transformations, and 
implications . since these products v7ould appear to be 
more important to complex problem situat:,ons. Though, 
these tests are specified by the SI model, they inc3,ude 
tests representative of almost all factors important 
to problem.-solving as cited in the review of empirical 
studies. 

Finally^ the population of interest will be fifth 
grade public school students. Children at the age' 
level associated with this ^rade have (generally mature 
cognitive skills. They are able to deal with verbal 
material with reasonable ease (McCandless , 1967) and 
probably lack the background of experience in solving 
problems "which might obscure the workings of problem- 
solving strategies. Thus problem-solving tests for this 
age level shou^ld yield a valid sample of problem 
solving behavior. 

In order to fulfill the purposes of this study 
the following specif ic objectives have been form.ulated: 
1. To adapt the verbal maze, the simulated 

problem situation, and concept identification 
tasks with both single and multiple solutions 
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to group administration with elenientary 
school students. 

From the test results gathered on a set of SI 
tests and the problem-solving criteria, to 
deter.T.ine the simple relationship between each 
of the abilities and the measures from the 
criteria. 

To determine a structure underlying performance 
on the ability measures and the criteria. 
To determine the predictability of performance 
on the criteria from knowledge of the abilities. 
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METHODOLOGY 

In this chapter the development of the criterion 
instruments will be described in terms of the samples 
usaci, the developmental procedures followed, the 
content of the instruments, the scoring procedures, and 
the collection of reliability data. Then the primary 
study will be described in 'terms of the sample employed, 
the instruments used, and the analyses carried out on the 
collected data. 

Crit erion development 

Samples . 

Two different sam.ples vere required. The first 
sample consisted of approximately 3 0 fifth grade 
students from a rural Indiana elementary school. 
This sample was employed in the preliminary testing of 
the criterion instruments. The second sample consisted 
of 14 0 fifth srade students from two upper middle- 
class elementary 'schools. These students comprised six 
classes, three in each school. This sample was used 
for determining the test-retest and alternate forms 
reliability of the criterion instruments. Two classeSj 
one from each school, were used in collecting data for 
each of the three types of instruments. 



^1 

Instrument developrr.gnt , 

Simulated problem situations ^ The simulated 
problem situations were based on the work done by 
HcGuire and others (McGuire and Babbott , 1967) at the 
Center for Educa-cionai Development, >University of Illinois 
College of Medicine in Chicago. The prototype of this 
instrument was originally used to measure physician's 
competencies in patient management situations. The 
•rationale for use of this type of instrument was that 
it provided problem situati.cns ' which could be relevant 
to the concerns of the population of interest and at 
the same time yield a number of measures of the problem- 
solving process. 

Four forms of the simulated problem situation 
were developed: a multiple solution problem, a si^.gle 
solution problem, and an alternate form for each* The 
problem situations chosen v/ere as follows: 

!• The- New Bike - multiple solution problem. 
In this problem the S is presented with a 
situation where he desires to purchase a 
bicycle he has seen in a store window* l\ 
nujnber of ways of obtaining the money are 
made available to the S, and he m.ust choose 
among these in such a way as to earn the 
necessary amount of money dn a minimum amount 
of time. , 
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F ree Ice Cream - niultiple solution problem-, 
alternate form. In thic form, the S is 
presented with a situation where the local 
ice-cream store ic offering a prize, of a 
year's wort/i of fr^ee ice--cream in an ecology 
drive to clean up the tovjn. In order to win 
the prize 5 the S must earn a given numjoer of 
points, points are earned by planting trees, 
recycling a variety of materials , or picking 
up trash in public places. Again a time limit 
is set . ■ 

The State Fair - single solution problem. 
In this form the problem situation is one where 
the S wants to go to the State Fair with 
friends, but must find the price of admission. 
The money is somewhere in the S's house, but 
the location is not known. The S must 'search 
through the house in order to find the money 
which is hidden, taped to the bottom of a 
dresser drawer in the S^s bedroom. 
The . Missing Friend - single solution problem, 
alternate form. In this form the problem 
situation is one where the S is supposed to take 
a friend to a club meeting after school so that 
t?ie friend may join the club. However, the friend 
is not at school in the afternoon. The S must 
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find the friend and take him to the meeting 
before the club adjourns for the afternoon. 

After choosing the problem situations, a problem 
jnap was created for each problem. Figure 6 details 
the problem map for The State Fair. The purpose of the 
problem map is to specify all the steps the S may take 
in- solving the problem , and the order that they may be 
taken. The problem map involves three different 
symbols. Squares denote information provided to the S. 
The hexagons denote a decision that the S must make. 
The triangles represent points in the problem where the 
S may quit or where a solution has been found. Arrows 
indicate .the direction of movement through the problem 
map . 

To indicate how this works, refer once again to 
Figure 6. The box entitled Start specifies that the 
S is given the statement of the problem; the S is told 
that he must find the money in order to' go to the 
State Fair. . Then the S proceeds to a decision point 
where he must decide to do one of three things: (a) 
look in the hall closet, (b) think of possible places 
the money might be, or (c) ask mother for the money. 
The first and third choice give the S additional 
information and direct him back to the decision point, 
with the instruction to make another choice. The S 
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Figure 6. Problem map for The State Fair. 
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proceeds in similar way through other information boxes 
and decision points until he reaches an exit box. It 
should be noted that in the cases of The New Bike , and 
Free Ice Cream that information boxes also sometimes direct 
the S to the fact that he has earned a certain amount 
of money or points, and that it has taken him a given 
number of days. It should be noted that the path taken 
by different S's will depend on the choices he makes, and 
may differ f rom S to S . 

The next step in the instrument development process 
was- to put the problem map into a format suitable for 
classroom administration. For this purpose, the problem 
map was divided into sections containing a decision 
point and one or several information boxes. These 
sections became pages in a test booklet. Figure 7 
is a reproduction of the first section from The State 
Fair. The initial paragraph gives the pertinent 
information for that section. After reading this, the 
S must make a decision based on this information plus any 
other information he may have collected. Decisions are 
made by choosing one of the alternatives listed at the 
bottom of the page. The box beside each alternative then 
gives the S feedback directions on which section to go to 
next, how much money or points he may have earned and' 
days he may have spent by choosing the alternative , or it 
tells him that he is finished with the problem. 
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Figure 7. Sample section of The 
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TrlE STATE FAIR 



Let's pretend that Gomewhere in your house you hav^ 
hidden a $5 bill. You have ^been saving it for so.^ething 
special. Now so^^ething special has con:e along ^ the 
State Fair starts today. There will be lots of rides 
and other fun things to do. Your friends are all 
planning to go to the fair this afternoon. You need 
the money so that you can get in with your friends and 
enjoy the rides. There is only one problem - you 
cannot remember where you hid the money. You are sure 
it is somewhere in the house b.ut you don't know exactly 
where it is. Which of these things would you do first 
in order to get the money to go to the fair? 



Look in the hall closet 
because closets are good 
places to hide things. 
Sit down and think of . 
some of the places you 
might have hidden the 
money. 

Ask your mother for the 
•55 since she just got paid 
two days ago. 



There is nothing in the 
hall closet but coats. 
Make another choice. 
So to' Section E. 



3. 



Your mother will not give 
you the money since it 
must be used to buy 
food. Make another 
choice- 



Figure 7 
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A special process was employed to provide the 
informative feedback given in the boxes next to each 
alternative. In order to prevent the S from reading 
all the informative feedbe.ck, and then choosing the most 
reasonable alternative, all information in the feedback 
boxes in each section was printed invisibly on the page. 
This was accomplished through the use of the Latent ' 
Image Process system manufactured by the A. B. Dick 
Company . In this method normal DITTO Masters are 
prepared with the visible printing, and then the 
printing intended to be invisible is placed on the master 
using regular master and a special imprinting sheet, 
tJie Latent Image Master . When the resultant master is 
run on a duplicating machine , the printing which is 
intended to be invisible is imprinted into the page, 
without becoming visible. A special pen, called the 
Latent I ma.^e Developer ren ^ is used to bring out the 
invisible printing. The pan is rubbed over the area 
where the printing is located and the printing becomes 
visible against a light yellow background. Since the 
S develops only those feedback boxes corresponding to 
the actual decisions he makes, a trace of the decisions 
which the S makes vhile solving the problem is preserved. 

The next step in the developmental process involved 
trying out the preliminary version of the instruments on 
a small group of fifth graders. Feedback was collected 
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on a variety of topics such as readability, clarity of 
directions, necessary time limits, workability of the 
Latent Ina^e materials, the amount of assistance needed 
by the Ss. The feedback information gathered from the 
first testing session was used to revise the instrument. 
The. most significant revision was the creation of a 
sample of the instrument to acquaint the Ss with the 
workings of the Latent Ima^e process and how to follow 
directions given in the instrument. This tryout and 
. revision process was carried out twice more after this 
first trial, before the instrument was judged ready 
for reliability testing. 

In order to score the simulated problem instruments, 
the order of .developed responses as they appeared in 
the test booklet i^as transferred to a mark sense 
scoring sheet. These score sheets were read, and punched 
cards were produced which contained an ID and a number 
of punched columns corresponding to the possible 
number of choices for that particular form. Thus for 
Jne, State Fair there were 3 0 colurans since the problem 
has 30 possible choices. Sach col^xmn contained a 1 if 
the corresponding feedback box was developed and a 0 
otherwise. These cards were submitted to a computerized 
scoring program which calculated a namber of different 
measures deperiding on the particular form of the test. 
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F^r tha two ,t,uitiplo solution instruments. The • 
New_3ike_and Free, Ic_ej:.r_eam , the scoring program calculated 
nine measures . The first cwo variables dealt .with the 
type of finish; the first indicated whether the S quit, 
stopped or successfully solved the problen ; the second 
indicated whether or not the S had met the conditions of 
a successful solution. The third variable was the ntmiber 
of dollars or points earned. The fourth variable 
indicated the number of days the £ used in earning 
the points or money. The fifth variable was a strategy 
measure. This was calculated by co.nparing the decisions 
made by the S with those necessary for a sure logical 
solution to the problem. The number' of decisions in 
common divided by the total nOunber of decisions in the 
sure logical solution times 100 became the strategy 
measure. The sixth measure served as an indication of 
the amount of information used by the S. The measure is 
also a ratio of. the S's decisions to seek information to • 
the total i^umber of possible information seeking decisions, 
multiplied by 100. The seventh calculated measure indi- 
cated- the number of decisions the S made which were 
inconsistent with the information the S possessed or 
direction, he was given. The score was the total number 
of inconsistent decisions the S made while working on the 
problems. The final measure derived from the instrument 
was an indicator of the total number of decisions the S 
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ir.ade in Golving tho rroblen. Due to the faot t:\c".t the 
S was allowed to pass Throush sever.-l decision points 
rcoro than once, the total nujnber of steps had to be in- 
ferred rather than actually counted. Those inferences 
a.-r-.~ec , that at the.vo- particular dccisi-n points the S 
alv.-c-.y? rerformed lojically and according to the p;iven 
cirr.ctionc:. 'Votal tine to solve the probleri vas also 
collec-red , 

?or the two cingle solution proble.r.s, The St ate Pair 
±^-LilriLgA'"w? :. r-r-iend . the scoring program yielded five 
neasurer.. The mearurerr conr^isted of the finish type, 
th^e srrateo, inconsistency, nu.-nber of steps , and tiuie 
scores. The calculations of these scores were exactly 
the sa-iie as describee in the previcjs paragraoh. 

L'SP^J^BiIA^S^'llBl' This sot of proble.T.-5olving 
criteria was adapted froni the tasks proposed by Kayes 
(1S.35). In the original tasks , ^s learned lists of 
Mired naT.es to a c^it-rion of three perfect trials. 
They were then told that the name pairs represented links 
between pairs of spies-. Th^-v v/ere then presented with a 
series of problcir. tasks in which the names of two suies 
were c-iven and the -J was ashed to find a' route to get the 
nies3a--e fro^n the first to the seccnd. Ss were required 
to car:r>- out these tasks usinj only what they could 
"work out in their heads . " 
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In order to adapt this type of task for use in 
element-ary school classroorr.s , it was necessary to 
develop a gro'up administered node. The objective was 
to develop four ^or.3 in this node: a sin.l. solution 
problen, a r.ultiplc. solution problem and an alternate 
fcrr. Of each. in order to design the.e forms, it v;a3 
necessary to develop the linkages between names which 
could again be portroycd as problem maps. The maps for 
the four problem are given in Figure 8. The nodes in 
these m.aps represent the spies v;ho are linked by the 
interconnecting lines. 

After the maps were specified, comj^on English first 
names were assigned to each of the nodes. These nam.e3 • 
were then formed into pairs ^ased on the linkages given 
in the m.r.ps. Test booklets were then constructed with an 
example, a single solution problem and a multiple, 
solution problem included. The format of each problem 
consisted of a brief introductory paragraph giving the 
name of the spy sending the m.essage and the name of the 
spy who was to receive it. Then followed a list of -spy 
name pairs which completely described the problem map. 
The rest of the ^.age consisted of lines of connected 
boxes. Figure 9 i, exa.mple of such a page. The S 
was told to use each line to attem.pt to find a solution 
path by writing connected spy names in successive boxes. 
If the S came upon a dead-end, he was instructed tc 

G3 
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restart on a nev; line. Finally 3s v^ere instrueted to 
write dovm the clock time v;hen they had found a solution. 
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Figure 8. Problem maps for the Verbal Maze problems. 
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Figure 9. Example of a Verbal Maze problem. 
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The first daveloped forms of these instruments were 
ad;ninisteree to the. tryout sample. Infornation was 
collected with respect to clarity of directions, ease 
of utility of tlm problcr^. format, and time requirements 
for completion of the task. The forms were revised twice 
with intervening tryouts , before the . instrument was 
judged ready for reliability testing. 

The verbal maze ins tru.T^nt yielded three measures 
for the combined single and multiple solution problems. 
The first of these measures was the total number of trials 
used in finding all solutions. The second measure was 
the total time taken to find all solutions. The third 
and final measure was the total number of solutions found 
in both the single and multiple solution problems. 

.qoncept Identification ProhK^r... . The primary objective 
in the development of concept identification tasks was 
to construct, a suitable format for group administration 
of concept identification tasks. First, however, the 
concepts had to be defined. In parallel with the other 
developmental efforts, four target concepts were identified. 
Exem.plars consisted of single abstract geometric forms with 
four bi-valued dimensions.-. The dimensions were: (1) 
Large - Small, (2) Ccuare - Circle, (3) Two holes - Four 
holes, (i;) White - Crosshatched . Two dimensions were 
relevant and two were irrelevant. The four target concepts 
consisted of two con jufte-five concepts and two exclusive 
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disjunctive concej-s. The conjur.ctive concepts „ere 
LURSs (Two hole. Squares) and FLIX'es (Four Hole Circles). 
The disjunctive concepts were DROOGs (Crosshatch^d or 

Square but not both) and ^t--^^ run^<^ 

anc (I'.'hr.te or Square but not 

both). 

"he presentation .ode .as n^odeled after the selection 
paradig. (Bourne, et al. , 19 71). Xhus all permutations 
of the exemplars were presented simultaneously on one 
page. Figure 10 details the form of each exemplar. 
Ihe letter identifies the .articular exemolar. 



H 



CO 



1 



YeE jTKIS IS HOT A FLpj I 



"{ I 



Figure 10. Typical exemplar for the conc-pt 
Identification task. 



Feedback was provided to the' S by means of the bo>.es 
following the YES and NO. For each exemplar the S , 
had to decide whether it was or was not an example of 
the target cocnept. If it was, the S chose the YES 

i>ox, otherwise the NO hov t>,^ j- 

^i.e uu box. The feedback was provided in 

^-tcrvt_jma,^^ form and the 3 indicated his decision by 
coloring in one box or the other with the Latent_Image_ 
2evcIoper_Pe^ The revealed printing told~^r7;~^r 
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. or not the exemplc- was or was not and example of the 

target concept. For each problem, one positive exemplar 

was chosen as a focus. In order to record the sequence 

of exemplars the- 3 considered, he v;as required to write 

the identifying letters across the top of the page in the 

order he looked at the exe.-nplars . Ss vere required to 

respond to all 16 exemplars. All four forms are presented 

t 

m the Appendix. 

The four forms vvere c-veloped and administered to 
the tryout group. Feedback vas collected on the clarity 
of the directions , the diffictiity of the problem.s, and 
usability of the problem format. It was found that the 
directions were quite difficult to understand, and' so 
sample problems were constructed to explain each problem 
and familiarize the Ss with the format. This revised 
form was tried out twice m.ore and revised before it was 
judged ready for the reliability testing. 

The concept identification tasks yielded four 
m.casures. In order to obtain these measures, the sequences 
of exemplars as they were considered by S and the decisions 
made were transferred to punched cards. These cards were 
submitted to a computerized scoring program which calculated 
three of these measures. The first measure was the number 
of errors in exemplar identification made by the S. The 
next two measures were based on the scoring procedures 
detailed in Laughlin and Jordan (1357) and were measures 
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of the foousiag and scanning behavior of the S. In 
order to calculate. these neasures it was necessary to 
ass-a.e that the „«ber of errors was equivalent to the 

u..„o „w S to .laentxfv the ta^g-t 
concept. The final measure -...as the total elapse, ti.e 
the S took to „ori< his „ay through the 16 CKenplars. 

Procedures. Two types of -eii=hn.-+,. 

j'i^co -cxxaDiiity measures 

were obtained for each of the three types of proble.- 
solving criteria. These were i..ediate alternate for.s 

reliability and two week t.-'^.t-r^i-o^- k-t^ 

u_..T reteo.. reliability. Table 

1 sununarizas the tyoes of rpi i ^s-n -•- • - 

yut,o or re.iia^ility intormation 

obtained for each form. 



^able 1 



Summary of the tyoes of re T •• i • ^ 

Obtained fo^ ^^^^i^^^^i^^^^^ 

Test-retest Alternate Forr.s 



Si-Tiulated problem situations 






1. The New Bike 


Yes 




2 . Free Ice Cream 


Yes 




3. The State Fair 


Yes 




The Missing Friend 


Yes 
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Table 1 (continued) 

Test-retest Alternate Forms 



Verbal Maze Problems 



0 * Sing 1 e g o 1 u t i cn v 1 


Yes 


Yes 


5: Multiple solutions 


n Yes . 


Yes 


7. Single Golution ^2 




Yes 


8. Multiple Solutions 


#2 


Yes 


Concept Identification 


Tasks 




9. Conjunctive LURBa 


Yes 


Yes 


10. Disjunctive DROOGs 


Yes 


Yes 



11. Conjunctive FLIXes ' Yes 

12. Disjunctive RIL(3s Yes 



In the first testing session, two classes (one 
from each of the sample schools) received The New Bike 
and Free Ice Creain problems, two classes received the 
single and fault iple solutions verbal Maze problems 
#1 and #-2 and two classes received all four concept 
identification tasks. Due to procedural difficulties. 
The State Fair and The Missing Friend were not administered. 
For each type of problem-solving criterion, the test 
administrator first introduced the problem tasks. Then 
a short example of each task was worked out with the 
class as a v/hole, to make sure that Ss understood the 
nature of the task and the mechanics of the testing 
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situation. £s were instructed to write doxm the clock ' 
time after they had. solved each problem. Testing 
sessions were limited to a period of 60 minutes. 

In the second testing session the test-retest 
data was. collected. The first two classes again received 
The ?;ew Bike plus two other proble;:: solving tasks. 
The second two classes again received the concept 
identification tasks with targets conjunctive concept 
LUP.B and disjunctive concept DROOG , plus two other 
problem solving tasks. In the third set of two classes, 
one of the classes again received the Verbal Maze 
problem single and multiple solutions §1 plus another 
problem solving task, while the other received the same 
verbal maze task plus The -State Fair and The Missi ng_ 
^^^^-^ •■ ' Again the testing sessions were limited to 
'60 minutes. Fowever, due to procedural difficulties, 
not all classes were instructed to write down the clock 
time upon completion of their problems. 

Analyses. The data was subdivided into test- 
retest and alternate forms reliability groups for each 
of the. problem- solving criteria. Pearson Product 
Moment correlation coefficients, means, and standard 
deviations were calculated for each of the test-retest 
and alternate forms groups for each of the measures 
listed below: 
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^* The Hew Bike and Fr ee Ice Cream 
!• Money or points earned 

2. Bays used 

3. Inf orrr^r^tion score 

4. Strategy score 
£• Inconsistency score 
6. . Nunber of steps 

B. The State Fair and The Hissing Friend 
1. Strategy score 
2 * Inconsistency score 
3. NuTT^ber of Steps 

C. Conjunctive Concepts LURBs and FLIXes 

1. Number of errors 

2. Focusing score 

3. Scanning score 

D. Disjunctive concepts - DPvOOGs and RILGs 
1. Huiaber of errors 
Verbal Kaze Problems 

1. Total time to solution 

2. Total nun^ber of trials 

3. Total number of solutions 

Summary , The reliability study of the problem-solving 
instruments completed the development portion of the 
project. The criterion instrum.ents had been developed 
and judged ready for use in satisfying the primary 
^ objectives of the overall study. 
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The_Pr pbiem Solving A bi lities Study 

In this section the sample, instruiP.ent preparation, 
and administration, procedures will be described. In 
addition, te.t scoring procedur^es and data analyses 
will be delineated. 

Sample. 

The sample for this study consisted of 490 fifth 
grade students from 18 classrooi^s. These classrooms 
were located in six schools of the Tippecanoe School 
Corporation. Ss came from a '.cide variety of SES levels, 
but generally shared a common cultural heritage. Ss 
were generally of average intelligence, but the sample 
covered a wide range of intelligence levels. The age of 
Ss averaged 11 years. 

Instruments . 

The instruments used in this study consisted of 
two sets. The first set was comprised of 17 tests drawn 
from the set of tests used by Guilford (1957) to 
establish the Structure of Intellect model. These are 
3umm.arized in Table 2. The second set of tests were the 
problem-solving, tasks aeveloped in the first part of 
this study. 
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Table 2. 



Measure 



A Classification of the 17 SI tests. 

^ ] Processes Products 

CMDNE UCRSTI 



Word Completion x x 

V/ord Linkage x x 

. Plane Flight Test x X 

Apparatus Test x X 

Remembered Relations X X 

Learned Infornjation x X 

Related Alternatives X . X 

Utility Test-Fluency x X 

-Flexibility X • X 

Controlled Associates X X 

Expressional Fluency x X 

Symbol Production x X 

Sentence Order X X 

New Uses X X 

Sequential Associations X X 

Unlikely Things ^ * X ' X 

Logical Reasoning X X 

Judging Object Adaptations X X 
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Word Completion. The test represents -the SI 
ability of cognition of semantic units, or verbal 
comprehension.' Merrifield et al. (1962) found tests 
of this ability were related to their prcblem-colving; 
measures. The instrument consisted of twenty words, 
and the Ss task was to write a short definition for 
each. To accomodate the developmental level of the pop- 
ulation of interest, the instrument had to be modified by 
substituting a less difficult set of words. However, 
the structure was maintained by using the same parts 
of speech and choosing words of the same relative 
difficulty at the fifth grade level, as the original 
words were at the adult level. The' reliability of the 
instrument was .82 (Sheridan Psychological Services, 
1972). The score was the number of correct definitions. 

Word Linkage. This test represents the SI ability 
of cognition of semantic relations, or the comprehension 
of verbal relations. This test was included as a 
parallel to Word Completion . Each item of the test 
consisted of two words, between which there was a semantic 
connection. The task of the S was to choose from a 
list of three words, that word which represented this 
semantic connection. To accomodate the developmental 
level of the Ss, several items were eliminated due to 
difficult words. Thus the test wag shortened from 30 
to 25 items. The reliability, using the Spearman-Brown 
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Prophecy Formula was .70 (Sheridan Psychological Services, 
1972). The score was the number of correct responses. 

Plane Flight Tes-c. This test should represent the 
SI ability of cognition of semantic systems, a factor 
sonetiraos called general reasoning. It was modelled 
after the Ship Destination Test - (Guilford, and Koepfner, 
1971). The ability Tiieasurcd by this test was found to 
be related to problem solving by both Werdelin (1966) 
and Bunderson (1967). S's task was to calculate the time 
it v;ill take an airplane to travel from one point to 
another on a coordinate map. Travel between any two 
points takes two hours. Several factors are introduced 
which can affect this travel time at successive stages 
in the problem. No reliability information was available 
on this test so that the comjnunality of .33 was taken as 
the lower bound of the reliability. The score was the 
number of correct responses. 

Apparatus Test . This test represents the SI 
ability of cognition of semantic implications. The 
ability measured by this test was found to be related 
to their measures of problem solving by Merrifield, 
et al. (1962). Eacli item names a familiar object, and 
S's task is to think of two possible improvements for 
the object. Four items were eliminated due to the un- 
familiarity of the objects. The reliability of the test, 
using the Spearman-Brown Prophecy Formula was .72 (Sheridan 
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Psychological Services, 1972). The score was the number 
of acceptable improvements according to preestablished 
criteria. 

Rerne mbered Relations . This test represents the 
SI ability of n-.emory for semantic relations, thatis, 
the ability to store and retrieve information abr .t 
verbal relations- The- test was included because both 
Stevenson, et al. (1968) and Bunderson (1967) found that 
memory for verbal associations was related to problem- 
solving behavior. The S is presented with a list of 
relations betvreen a variety of objects and is told to 
study it for 2.5 minutes. The S then turns the page and 
answers a series of multiple-choice questions concerning 
the relations on the previous page. Due to problems 
with time limits, the test v;as .reduced from to 20 
itcmr^. The reliability of the Test, using the Gpearman- 
3rown Prophecy Form ula_^ was .U6 (Sheridan Psychological 
Services, 1972). The score was the number of correct 
responses. 

Learned I nformation . This test represents the SI 
ability of memory for semantic systems. It t,ras again 
included because of the findings of Stevenson et al. 
(1968) and Bunderson (1967). The S was presented with a 
set of paragraphs and given 5 minutes to read and 
memorize it. Then the 3 wars told to turn the page and 
reproduce the paragraphs in the order given, to the best 
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of his ability. At the top of the second page was 
a list of keywords to stimulate recall. Scoring was 
based on the correct order of the reproduction of the main 
ideas of each paragraph, with a maximum score of IS points 
Since the original paragraphs concerned the SI model and 
the vocabulary was of high difficulty, it was decided 
to substitute a parallel set of paragraphs with a 
vocabulary suitable for fifth grades. The reliability 
of the original -instrujnent was .64. Since no reliability 
was available on the revised instrument, the communality 
of .23 was used as a lower bound of the reliability. 

Related A lternatives. This test represented the 
SI ability of memory for semantic implications. It 
was included for the same reasons, as the other two 
memory tests. In this measure the S was given a study 
page" which contained a list of surnames associated with 
occupations, and given 2.5 minutes to study it. Then 
the S was told, to turn the page and respond to a series 
of. multiple-choice items. Each item gave a surname and 
the-S was required to choose from a list of four objects, 
the object which was associated with that person's 
occupation. The score was the number of correct 
responses. Due to several items which contained 
occupations quite unfamiliar to fifth grade students, 
10 items were eliminated. Thus the test-was reduced from 
30 to 20 items. The reliability, using the Speai-man- 
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Brown Prophecy Formula, was .ii9 (Sheridan Psychological 
Services, 1972). 

Utility T e£t. This test represented two different 

SI abilities, divergent prodtiction of semantic units and 

classes. These are more con^n-.only called ideational 

fluency and spontaneous flexibility. Kerrifield et. al. 

(1962) reported that divergent production of semantic 

units was related to their problem-solving tasks. 

Haroctunian and Tate (1950) found that spontaneous 

flexibility was correlated with their particular measure 

of problem solving.' Thus, this test was included in the 

study. The items are of a very simple nature. The 

S is asked to think of as, many uses as he can for a brick 

and a wooden pencil. The responses are scored for number 

of relevant responses, a measur'e of ideational fluency.' 

■They are also scored for the number of different ideas the 

S produces; a measure of spontaneous flexibility. The 

reliabilities of these measures are respectively .74 and 
.64. 

Controlled Associations "T^.^t. This test represents 
the SI ability of divergent production of semantic 
relations. Merrifield et al. (1962) reported that this 
ability was related to their measure .of, problem-solving, 
and thus it was included in tne study. In this test,' the 
S was given eight words and told to write down as many 
words as he could that meant the same or about the same 
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as the given words. The score was the total number of 
words that did, in fact, have a similar ir.eaning as the 
given words. The reliability of the instrument was 
given by She-idan Psychological Services (197 2) as 
• 7 C • ■ 

5xpr^e_s_sJ^ona]^ This test reprssents the 

SI ability of divergent production of semantic systems. 
It was included in the s-.udy in order to discover if 
the higher order divergent production abilities were ' 
related to efficient probleTn solving. The form of the 
task was that the S was given a sequence of four letters. 
Ke was then asked to produce as many sentences of four 
words as he could, in, which each word started with one of 
the four letters and the words were in the same order 
as the letters. The S<s score was the number of sentences 
he cDuld produce which met the stated criteria. Sheridan 
Psychologica:. Services (1272), give the reliability of 
this test as .67. 

Symbol Production. This test represents the SI 
ability of divergent production of .semantic transformations 
This was included for the name reason as Expressional 
Fluency, In this test the S was given a series of brief 
statements such as '"Airplane takes off". He was then 
asked to produce an abstract symbol for "airplane" and 
another abstract symbol for "takes off". Due to time 
limit considerations, the instrument was reduced from ♦ 
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51 to 33 items. The S's score was the number of symbols 
which were abstract, and in the judgement of the scorer, 
represented the object or phrase. Reliability, using the 
Soearnan-Erown Prophecy Formula^ v^-as .77 (Sheridan 
Psychological Services, 1972). 

Sentence Order. This test represented the SI 
ability of convergent production of semantic systems. 
It was included because Merrifield et al. (1962) 
reported that higher order convergent production abilities 
were related to problem, solving, but had not included this 
ability in their study. In this test, the S was 
presented with sets of three sentences about a particular 
subject or event. The S was required to determine the 
proper order of the sentences by numbering them in 
the order they should appear. The S's score on this test 
was the number of ' sentence triples he ordered correctly. 
Sheridan Psychological Services (1972) gives the 
reliability of this test as .56. 

New Uses. This test represents the SI ability of 
convergent production of semantic transformations. 
Merrifield etal. (1962) reported that this ability was 
related to their problem-solving measures. In this test 
the S was presented with a picture containing a number 
of. objects in a given setting. Below the picture were 
listed a number of functions. The ' S was asked to find 
objects in the picture which could fulfill the lis-ced 
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functions, with the restriction that the function would 
be a new or unusual use of the object. The S's score on 
the instrument was the number of items to which he 
responded successfully in accordance with the given 
criteria. The reliability of the test, as given by 
Sheridan Psychological Services (1972) was .47. 

Sequential Associations. This test represents the 
SI ability of convergent production of semantic implications. 
Merrifield et al. (1962) also reported that this ability 
was related to their problem-solving measures. In this 
test, the S was given sequences of four words. The task 
was to number these words in such a way that there 
existed a semantic connection between suceeding pairs 
of words. The S's score consisted of the number of 
sequences -which were correctly ordered. The reliability 
of the test was .75 (Sheridan Psychological Services, 
1972). 

Unlikely Thinp:s . This test represents the SI 
ability of evaluation of semantic systems. It was 
included in the battery because both Karootunian and 
Tate (1960) and Bunderson (1957) reported that judgement 
or evaluation was significantly related to their problem- 
solving measures. In this test the S was presented with 
a series of pictures, in which two or more things were 
out of place, incongruous, or unlikely. Beside each 
picture four alternatives were given. The S's task was 
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to choose the two alternatives which were most unlikely, 
incongruous, or out of place. The score was the number 
of correctly chosen alternatives. Sheridan Psychological 
Servipes (197 2) gives the reliability as .5U. 

Logical Reasoning. This test represents the SI 
ability of evaluation of semantic implications, or 
verbal reasoning. Three studies found that this ability 
was related to their problem-solving measures; Karootunian 
and Tate (I960), Werdelin (1966) and Bunderson (1967). 
The original, form of this test was too long and the 
vocabulary too difficult for fifth graders. In order 
to adapt the instrument, every other item was deleted 
from the original form. Then parallel items were 
constructed, so that the form of the logical proposition 
was kept intact, but the vocabulary was greatly simplifed. 
Thus • each item remained a logical premise with four 
alternative conclusions from which the S must Vnoose 
the one which logically follows. The instrument was 
consequently reduced from 4 0 to 2'0 items. However, 
S's score remained the number of correctly chosen 
alternatives . • The Spearman-Brown correction yielded a 
reliability of .72 for the shortened test. However, 
since the content of the items was changes the communality 
of .35 was taken as the .lower bound of the reliability. 

Judging Object Adaptations. This test represents 
the SI ability of evaluation of samantic. transformations. 
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It was included in the test battery because Harootunian 
and Tate (1950) found that a judgement factor was 
significantly related to their measure of problem- 
solving. In this tent, each item consists of an object 
naxe and three alternative uses. The S was instructed 
to choose the alternative -vhich was relevant and the 
most unusual, ingenious, or clever. Due to the un- 
familiarity of some of the objects names, five items 
•were deleted from the test form. Similarly, the directions 
required extensive rewriting to bring the vocabulary 
down to the level of fifth graders. The S's score was 
the number of correct choices' he made. The reliability 
of the instrument, employing the Spearman-Brown formula, 
was .U2 (Sheridan Psychological Services, 1972). 

Problem Solving Criteria . Five problem-solving 
criterion instruments were employed in this study. 
These were Jhe Jieu^_Bike^ Th-s State Fair , LURBs ^ DROOGs, 
and Verbal Mazes single and multiple solutions #1. 
These instruments- were thoroughly described in the previous 
section on criterion development. The measures yielded 
were as follows: 

1. The IJew Bike - number of dollars earned, number 

of days used, information score, 
strategy score, inconsistency 
score and numbeir of steps. 
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2. The State Fa ir - strategy score, inconsistency 

score, and number of steps. 

3. LURBs - number of errors, focus score, scanning 

score, and total elapsed time. 

4. DROOGs - number of errors and total elapsed 

time . 

5. yerbal_Mazes - total time, total trials and 

total number of solutions. 

Reliabilities of these measures are given in the RESULTS 
section. 

The above 22 tests yielded 31 reliable measures. 
These measures became the basic data for analysis in 
this study. 

Procedures . 

After the SI tests had been revised, the 22 
instruments , were assigned to one of five one hour testing 
sessions. Arrangements w6re made with each of the 
participating schools for fiv*e one hour testing sessions 
in each of their fifth grade classrooms. These sessions 
were spaced at one week intervals, over a period of . 
five weeks. 

Eight test administrators were recruited, and each 
was assigned to one or more classes. These persons 
received pretraining in general test administration 
procedures and were acquainted with the goals of the 



project. On each Friday preceeding a week of testing, 
the administrators met and were introduced to the next 
week's tests. They were acquainted v/ith the mechanics 
of each test, and possible testing problems were 
discussed. In the following week those tests were admin- 
istered to Ss in the sample. During each testing 
session, the administrator kept a written log containing 
anecdotal information about the progress of the testing 
and appropriate timing information. Table H gives the 
sequence of tests as the Ss received them. 

After all testing was completed, the instruments 
were scored. In the case of the problem-solving 
criteria, the test responses were transcribed directly 
to data sheets by clerical personnel, and then prepared 
for computer scoring. For scoring the SI tests, three 
scoring judges were recruited. The 17 tests were dividled 
among these three people and each one became exclusively 
responsible for scoring the assigne.d tests. The 
procedure for scoring each test involved three steps. 
The scorer first became familiar with the test"^"^ey and 
scored a sample of about ten tests. The scorer then 
went over these tests with the person in overall 
charge of the project, to insure that the tests were 
being scored properly. Then the scorer finished the entir 
set of tests before moving on to the next set. 



In cases where test scoring required judgement 
upon the part of the scorers, the reliability of the 
judging was assessed. After an entire set had been 
scored, a random sample of 20 tests were drawn and 
rescored by the same person. These sets of rescores 
were then correlated with the original scores to 
determine the reliability of the judges. 



Table 4. 



Test administration sequence and timing 
xnformation. ° 



Session 



Instruments 



Administration 
time (min.) 



Utility Test 
Controlled Associates 
Verbal Haze' frl ■ 

lixpressional Fluency 
Remembered Relations 
Apparatus Test 
Unlikely Things 

Sentence Order 
Sequential Associations 
Judging Object Adaptations 
New Uses 
VJord Linkage 
The State Fair 

Word Completion 
Plane Flight Test 
Learned Information 
Logical Reasoning 
Concept LURB 
Concept DROOG 

Symbol Production 
Related Alternatives • 
The New Bike 



10 
12 
35 

8 
9 
14 
10 

8 
6 
8 
3 
6 
15 

7 

8 

7 
10 
15 
15 

a 

9 
US 
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As each S^s SI test was scored, it was recorded on 
a data card for that S. Each S was assigned a unique 
identification number which was recorded on each test 
booklet. All this information was then combined with 
the computer scoring output to make up individual cases 
of data for each S consisting of all his test scores. 
Finally, this information was placed on computer cards 
and only those cases with complete information were 
selected out for analysis. 

s - 

Analyses . . - 

The analyses for this stydy were aimed at providing 
ansvjers for three barsic questions: 

1. What are the simple relationships among the ai 
measures derived' from the SI and problem- 
solving tests? 

/ 2. How predictable are the performances on the 

problem-solving measures, given the SI tests? 
3. VJhat is the nature of the underlying structure 
of the CI tests and problem-solving criteria? 

Information regarding the first question was derived 
from the simple Pearson Product Moment Correlation 
coefficients among the 31 measures. First, descriptive 
statistics were generated for each of the measures. 
These included means, variances, standard deviations, 
ranges,, and skewness indices. The skewness index was 
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examined for each variable and suitable transforations 
"ere carried out for those variables which were highly 
Skewed, using the set of variables, where the transformed 
variables replaced their counterparts, the correlation 
coefficients were calculated. 

In order to obtain information regarding the second 
question, stepwise regression analysis was performed 
for each of the problem-solving measures: The computer 
procram used for these analyses was the REGRESSION sub- 
program Of the Statistical Package for the Social Sciences 
(SPSS! Nie. Bent, and Hull, 1970). For each of the 
problem-solving measures, this program yielded a multiple 
P. and r2 for each step, the change in r2 f^o„ ^^ep to 
=tep, and the order of entrance of predictors into the 
regression equation, m order to determine the significant 
set of predictors for each problem-solving measure, an 



F test, 



F s ' - m., - 1) 



for the change in from step to step was applied. 

is the multiple correlation for equation 1 which contains 

predictor variables. is the multiple correlation 

for the second equation with m^ predictors, atd N is the 
total nu^nber of cases in the sample (Draper and Smith, ' 
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1966). The significant set of predictors was the set 

from the last step in the stepwise regression analysis 

2 * 

which caused a significant increase in R *at the 5% 
level. The final was then interpreted as the amount 
of criterion variance accounted for by the SI tests in 
the significant predictor set. 

The approach to obtaining information concerning 
the third question^ that of underlying structure ^ was 
threefold. Firsts a conventional factor analysis was 
carried out. An iterated principal factor solution 
(Harman, 1967) was calculated from the basic correlation 
matrix ^^^pg^^* This yielded 27 facto^ with positive 
roots. These eigenvalues were plotted against the factor 
number as specified in the Scree Test (Cattell, 1966a, 
1966b). The scree slope was judged to begin at the sixth 

factor, and thus the first five factors were considered 

* 

to .be relatively free of error variance. These five 
factors were then orthogonally rotated by the varimax 
procedure (Harman, 1967). The resulting solution was 
judged not completely adequate due to a low hyperplane 
count in some f actors ' and ._f actor overlap on several 
tests. An oblique Promax rotation (Hendrickson and VJhite, 
1954) was then carried out. In this procedure, a varimax 
solution with elej^nts a^^ is used to generate a target 
matrix W with ellments w. . = a^. ./^'*'^/a. . . Thus each 
element of W is the corresponding element of the varimax 
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solution raised to the Mth power v/ith the original sign 

retained. A least squares trans forinat ion is then calculated 

between the varimax solution and this target matrik 

vv by the "Procrustes" technique due to Hosier (1939) 

and Hurley and Cattell (1S52). The resulting transformation 

is then applied to the original varimax solution, to 

yield an oblique reference structure. This structure 

was then used to compute the primary factor loadings, the 

intercorrelations' among the primary factors, and the 

test - factors correlation matrix using the formulae 

found inr-Harman (1967 ). 

The second approach taken to the question of 
underlying structure was to perform a canonical correlation 
analysis (Hotelling, 1935). The first set was the SI 
tests and the second was the problem-solving criteria. 
First, canonical roots were extracted and correlated 
linear functions of the SI rests and the problem- 
solving criteria were calculated. In order to evaluate 
the relative importance of the resulting canonical 
variat^es, the redundancy for each pair of linear functions 
as defined by Stewart and Love (1968) was calculated. 
The redundancy is a non-symmetric index which indicates 
the proportion of variance extr^acted by a canonical 
factor of the first set which may be predicted from the 
corresponding canonical factor of the second set. . To 
calculate the redundancy coefficients for each set of 
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linear functions, the test-canonical factor- correlations 
(i.e. the canonical factor structure) had to be calcu-lated. 
This was done by multiplying the matrix of intercorrelations 
of the SI.„.t,ests (R^.) by the coefficients of the linear 
functions for the first set, A similar process was 
carried out for the linear functions of the problem- 
solving criteria. This yielded a set of factor loading 
vectors s^, for 'the SI tests and a similar set t^ 
for theproblem-solving criteria. The proportion of variance 
extracted by each canonical factor of the first set 
was given by the exp9?cssion s^s^/p where p is the 
nurnber of tests in the first set, A similar expression 
involving the t^^s and the number of variables in the 
second set gives the proportion of variance extracted 



by each factor in the second set. The redundancy for 
the i^^ func 
the formula 



the 5.^^ function of the first set is then calculated by 



1 p c^ 



where R is the canonical correlation for the first 
set of linear functions. Again, the redundancy for the 

• th 

i ' function of the second set is • calculated by a similar 
formula. These redundancy measures were then used to 
evaluate the importance of the pairs of linear functions, 
^he third approach in iiayestigating the underlying 
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structure was to determine the relationships of the 
problem-solving criteria to the SI structure represented 
by the SI tests. In order to accomplish this, it was 
necessary to attenpt to force the SI tests into in- 
dependent factors, as postulated by the SI model. This 
was . accomplished by extracting all factors with positive 
roots from the SI tests intercorrelation matrix by 
the iterated principal factor methpd. This yielded 13 
factors which were then rotated to a varimax criterion 
to achieve maximum uncorrelated simple structure. In 
order to study the relationships between the problem- 
solving criteria and the SI tests, this factor structure 
was then extended to include the problem-solving criteria 
by the method of Dwyer (1237) and Kosier (1938). The 
procedure is specified in the following formula: 

V = R' . F (F'F)"^ 
ps-si ' 

where V is the matrix of extension loadings of the 

problem-solving criteria on the SI factor matrix, R' 

' ps-si 

is the intercorrelation matrix of the problem-solving 
criteria with' the SI tests. F is the varimax rotated 
factor loading matrix of the SI tests. These extension 
loadings were interpreted as estimated factor loadings 
for the purposes of elucidating the inter-battery 
relationships. 
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Summary 

This concludes the cliapter describing the methodology 
of the project. The development of the problem-solving 
criteria has been described and the study of their 
reliability has been delineated. The instruments 
employed in the primary study have been described, and 
the administration procedures have been outlined. 
Finally, the analyses of the data from the main study 
of problem solving, were described in detail. In the 
following chapter the results of the analyses will be 
presented. 
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RESULTS 



In this chapter the results of the two studies ' 
will be reported. First, the results of the reliability 
study will bs presented. Then, the results of the 
prirr.ary study of proble.m solving will be reported. 

Reliabilit y Study 
The reliability study attempted to determine the- 
reliability of the problem-solving criteria developed 
in the first part of the study, by two different 
methods. The first method sought to establish the 
reliability of the ij^ar^-^.^ir.ents by administering them 
to the same group of subjects twice, with an interval 
of two weeks between administrations. The means and 
standard delations for each of the measures derived 
frdm the instruments for the first and second testing 
sessions are given in Table 5. In addition, the 
distribution of finish types and legal finishes for 
The New Bike are reported in Table 6 . 

For Thejiew Bi3ce only 17 of 42 Ss indicated that they 
solved the problem of the first bccasion and 5 of these... 
were not justified, while in the second session 26 Ss 
successfully solved the problem. Since the inconsistency 
score was extremely positively skewed, .a log^^ trans- 
formation was applied, and these are the means reported 
in. the table. 
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Table 5. Two week teat-retest means and standard 

deviations for the reliability ctudy of the 
problem-solving criteria. 



Session 1 Session 2 

Measure Moan S.D. Mean S,D, 



The New Bike N«'^2 

Honey :»arned 2^1.57 9.78 26.7^1 9.20 

Days used 9.i\3 5.12 il,60 '^•Sl 

Information 

score 55.93 15.93 56.19 17.01 

Strategy 

score 35.67 1^.83 35-76 1^.15 

Inconsistency 

acore .ii3 .90 .117 .93 

Number of 

steps 28.05 9.28 28.76 9,A8 

Concept LUH3 N*^^8 
Number of 

er^rors 4.58 2.91 4.92 3.28 

Pbcus score 2.98 2.20 3.85 2.32 

Scanning score I.09 1.21 1.46 I.23 

Concept DROOG N«48 

Number of errors 4.90 . 2.15 5.46 2.88 

Verbal Maze #1 N«46 
Total time 9|19 ^.31 5.67 3.23 

Total trials 6.20 2.25 5.11 1.92 

Total solutions 3.39 2.06 3.39 1.96 
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Table 6. Distribution of finish types and lecal 

finishes for simulated problen situations. 





Finish Type 


s 


Legal 


Finish 




Form 


Solved 
Problem 


Quit 


Did not 
finish 


Yes 


No 


The :;ew Bike (1) 


17 


8 


17 


37 


S 


The New Bike (2) 


33 


f 

^ ... I 


3 


35 


7 


Free Ice Cre.^m 


37 




7 


17 


27 


The State Fair 


13 


7 


3 


The Missinj^ 
. friend 


13 


3 


7 
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Two particular pieces of information should be 
taken into consideration regarding these means in 
Table 5. First, test administrators consistently 
reported that few Ss understood the nature of the 
disjunctive concept DROOG which they were supposed to 
identify. Secondj it should be noted that ^ the Verbal 
Maze >>'l was originally designed to have a maximum total 
of three solutions. Yet Ss discovered^ new solutions 
that met the criteria stated in the instructions, thus 
a mean total solutions of 3.3S. 

The test-retest correlations for these instruments 
are presented in Table 7. For The New Bike , the highest 
test-retest reliabilities were demonstrated by the 
information and strategy scores (r^^ = .50) of The 
New Bike > The other measures had reliabilities ranging 
from .19 (inconsistency score) to .48 for the number of 
stepis measure. "The measures for concept LURB demonstrated 
a wide range of test-retest reliabilities. They ranged 
from a high of .52 for number of errors to -.07, which 
■%as a not significantly different from zero, for the 



scanning score. The test-retest correlation of number 
of errors- -for concept DROOG was ^Iso non-signifleantv: 



However, the reliabilities for Verbal Maze #1 were all 
significnat and ranged from .36 for total time to a 



respectable .74 for total solutions. 
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^' • ^UV"" ''''-'^'-^^ reliabilities - '^oUaM-ty 



The New lilk o ^ 

I)ays used 
Inforir^atlon score 
Strategy Gcore 
InconslGtoncy score 
Nunber of steps 

iVunber o.f errors 
Focus score 
Scanning scoro 
Concept D PvOQG 5 

' Number of errors 
V erbal Maze 
Total tlrro 
-otal trials 
Totr.l solutions. 



.33 

.32 
.50 
.50 

-IS- 

.52 

.07 fn.s.)^ 
• 35 



^?!crie^^r '''' ^^^^^^^-^^tly different from 0.0 at t^e 
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"She secorc ar-rcach t-ke- tc c:rsessinc the r-liabiUty 
. .o.f-the probieR-solvins criteria w^s to compare the 
porfornanca of the .s^a.To. Gs on two supposedly al-ornatc , 
iorr-r of r.^ : nrtr-nont . T^e alt-r-:.t- forr.s for the 
3:.".-ictecI pro:;lt::,T jituAtions wore The :;ev; hike and Free 
multiple 3o:.ut;.cr, probler.s and The Stat e Tair 
:-:i^JlilsJr^r^yxlcrd sir,3lo -ol-.ticr. problems . The 
alternate forrr.s for the concept problejas v?ere the 
I^-df, ^^-^^ ILlXes conjunctive ccn::epts and the DR_C06s_ ■ 
a-id ci-junctive concn-r:. ri-:.ai:.^. "c^ie alternate 

fcr-r.3 of the verbal ::,aees were Verbal X.aze and Verbal 
I-'-?.::e The ^.eanr: and ^t.-.ndard deviations for these 

ir-.tri-.T.erits are presented in Table 8." In addition the 
finish types and le.<:al fi^ishe- v;h.crG appropriate for the 
four 'orr-c cf the simulated problcir^.s, are givan a-ain 
in Tahie 6. Frorr. this tahle, it is evident that both 
The.:;^:^ Bike _ and I::£r_~£^_Jlll^^. were quite difficult since 
only 12 3f successfully solved the first and only IC 
rolved the t-econd. The State Fair -Tind The Missin;^ ^^i^nd 
appeared to be scrr.av;hat easier since 13 out of 23 solved 
each of these. 

The alternate forn rri^ans of the simulated prohler. 
••ituationc. carnot be directly conparcd since the fornia ■ 
hove different nur.Lerc' of cections .-n.d -are only Gtructurally 
sijrilar. The concept probler-r, or the/t>ther han-^ c-^n 
.03 ccT-paroc 'j.tncc tnc concept pairs are of eaual 
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Table 8, 



•Voano and r.tandard deviations- for the altematf. 



Form 1 
"ean s.d. 



oi'-uiate-J problcrs ^he .\'ev.- El ke 



Nonrjy/point.-j carn-ici 2'i.65 
Days use-S o . 35 

.Tr.rorrration score 55.66 
Strategy score 35.25 
Inconslstorcy score .^Jl 
Number of .Steps 2 7.86 



9.60 
5.01 
15.9^ 

.88 
9.23 



Stratee:y score 
Inconsistency score 
.Vumber of steps 
Concept Tasks 
NiTibor of error.-. 
Focus score 
Sca.'^nlns score 

J'urr^bor Of errors 
Verbal Maze problems 

Total tine 
Total trials 
Total solutions 



I^jgL-S^g-te Fair 
75. 3e 23^0>i 
.82 1.05 
7.35 

LUR33 
".67 • 2.86 
3.06 2.16 
1.1^ 1.21 



9.15 
6.11 

3.n 



2.17 



^.35 
2.20 
2.01 



23 



^7 



i.5 



Form 2 

Hear, 



S.D. 



Free ,Ice Crean 



32.85 
5.23 
37.32 
30.68 
.92 
17.93 



15.91 
2.73 
18.93 
13.69 
1.07 
n.91 



T he Missing ?rlond 

3'(.'<6 12. ^6 

2.35 .18 

8-91 1.65 

".85 3.1^ 

^.02 2.79 

1.84 1.63 



5.00 



5.72 

2.80 



2.i)9 



3.30 

l.Sl 
1.85 



id 
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con-.pleKity . The mean nurrtLor of errors for conjunctive 

<if.G7) and TLIX-a (i>.35) vr-re hignly sir.rUar. 
i.V.vover, th- focurj and scanning score means for the 



y".I/:'j:; -..'ov,., - then t;io~- for tiK. 



For the Ver^.l l-^zos and .2, the n-aans for all Three 
:f.s>-ares v;er- 1-3-0 for tnc /2 for- than for the r'l 
fcr-n. It shcuLc. ue- noted tnat for ail alternate forms, 
tne tirst nentiorcic for,-?, v.-is adr.iinistered irjneciately 
before the ^second r.entioncr;. form. 

The alternate for^-r. reliaoiliry coefficient? for 
eac.i of the instruipen-cs ic^, presented in Table 2. These 
reyalts cast consiceraole dou:,^t on the assertion that 
the differerit fcrr.s of the instru-lents were, in fe.ct, 
alternate forns. Ivelvc out of the 15 reliability co- 
efficients were not significantly different frorr: z»ro. 
The reraaininn coefficients ranged fror: a high of ,B'4 for 
Verbal Kaze total solutions to a low of .34 for Verbal 
Maze total tirr.e . ' ' ' 

The results of tnis reliability study were used to 
Choose the criteria for the primary study of problem 
solving. It should ce obvious from Tables 7 and 3 
that the obtained reliabilities of both types were 
far from ideal, iiornelly, acceptable reliability 
coefficients lie in the ran^e of .70 to .99, which 
excludes every measure except Verbal Maze total solutions. 
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. Table 9. Alternate form reliabilities - Reliability study. 



Measure N r. 



Zr\o Knv; B.1.k=:> - P'rce Ice 
Ci»oarn 








Money/point? earned 




-.02 


1 

(n.s.)-^ 


Day? uaecl 




-.01 


(n.s. ) 


Ir.formntion score 




-.19 


(n.s. ) 


Strate^3:y score 




-.03 


(n.s. ) 


Inconsistency score 




-.08 


(n.s. ) 


Number of steps 




-.04 


(n.s. ) 


Concepts LUR3-FLIX 


43 






Number of errors 




.56 




Focus score 




.20 


(n«s. ) 


Scannlnr; ocoro 




-.23 


(n*s. ) 


Concent. '3 DROOG-RILG 


47 






iNur.ber of Errors 




• 19 


(n.s. ) 


Verbal Mazes ^1 - 


45 






Total time 




.34 




Total trials 




.46 




Total solutions 




.82 




The State Pair - I'he 
Missinf: i'Yiend 


23 






Strategy score 




-.03 


(n.s. ) 


Inconsistency score 




.00 


(n.s. ) 


Nurrber of steps 




.21 


(n.s. > 



•''Correlation is not significantly different from 0*0 at 
the .05 level. " 
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' 0 0 

Ho;vever, it was decided that for research purposes 
that all measures v;hic]i erdiibited at least one type 
of reliability coefficient different from zero would be 
usei ir. the prir.i.7.ry study. The rationale for this 
decision was based on the assertion p.ade by KcGuire and 
Dabbott (1967 ) and 'Iref finger and Poggio (1972 ) that 
for .'nany kinds of problem-solving tests, conventional 
measures of reliability were inappropriate. This can 
be due to nuir;ber of causes, including differential 
learning across a number of occasions of solving the 
sare problem, or differential transfer from one task 
to the next. The result of this decision was that 
Concept JROOG and The State Fair were almost entirely 
eliminated and the scanning sc.ore from concept LUI^S was 
also deleted. 
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In primary study, the data for analyses were derived 
fron 21 .noasurcs r.-.thered on a sa-,ple of 490 subjects. 
Or.ly tho3e v.- ;,:th ccnplete data on all variables were 
.. selected for analyses . Table IC reports the descriptive 
statistics for each of the' variablesincluding the 17 
SI tnsts and the four prol^len^.-solving instruments. This " 
table gives the mean, standard deviation, range, skewness, 
B.nd reliability for each of the variables. Due to 
extre.Tie positive skewness {5.67) cf the inconsistency 
score from Z^}eJ^ew_2ike^ the variable labelled in- 
^ ^ consistency score in the table is actually the log base 
10 of the inconsistency score. 

Intra -• ud c:e _r e_]^ ah j_l_2_t v , 

The first analysis that was carried out was to 
determine ti.-e reliaoility of the scorers on those tests ' 
in whicn scoring required substantial judgement. Lach 
test v^as scored by only one individual. Table 11 
presents the score-rescore correlation coefficients 
for each of tliese types of tests, All coefficients 
were within a ran-e of exco.llont scorer reliability 
(i.e. £:reator than .?0) except for b ymbol Productio n ' 
(.79). However, tiiis last was judged to be sufficient 
for research purposes. 
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Table 11. 



Jo^''^?^;1l1/f,^^'-?^^"y determinations 
.JudeoLi-!: ' ^^'^^-"^^^"S substantial scorer 



Measure 




Hescore 








. ^'1 


i 




S.D.2 


12 


^12 


U'i'FLU 


-3-35 


6.51 


14.85 


7.62 


n«3. 


.94 


UTFLX 


4.10 


4.28 


4.2 


4.6 


n.s. 


^.57 


CA 


11.00 


6.23 


11.00 


6.32 


n.s . 


.93 


EP 


2. 40 


1.98 


2.35 


1.6.3 


n.s . 


.93 


AT 


11.00 


7.71 


10.83 


8.20 


n.s* 


.98 


SO 


5.30 


2.69 ■ 


5.30: 


2.69 


n.s. 


1.00 


SA 


2.95 


1.40 


2^95 




n.s. 


1.00 


Nv; 


8.90 


4.99 


3.95 


^.80 


n.s. 


.99 


WC 


8.90 


4.89 


8.75 


^1.82 


n. 3. 


.99 


I*! 


9.20 


4.00 


9.70 


3.96 


n.s. 


.95 




14.05 


5.34 . 


13.45 


^.A7 


n.s. 


.79 
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Vari able Intercorrelation s . 

The Pearson Product lloment intercorrelations 
for all the possible pairs of variables are reported 
ir. Table 12. For this table, any correlation greater 
than .11 is significantly different from zero at the 
S^o level. For purposes of describing tne results the 
table will be subdivided into three sections, the SI 
test intercorrelations, the problem-solving criteria 

intercorrelations, end the -fao^- ^^^^-^ ^ . 

^5 ciiiu T:iie ;:>x test-problem-Golving 

criteria intercorrelations. 

^ue mean mtercorrelation of the, SI teGts was 

.33 v:ith a range fron .10 to .Gl. Thus -rail the SI 

tests were positively correlated to a moderate degree. 

This positive manifold increased the possibility of 

underlying common factors. 

for the problem-solving criteria, the mean 

magnitude of the intercorrelations was .12 with a 

ranee of correlations from -.28 to .94. i+4 of the 78 

correlations did not exceed the value (.10) necessary to 

be significantly different from zero. The largest 

intercorrelations appeared among m.easures from the 

same instrum.ent as would be expected. The negative ' 

correlations are also to be eynectf^ri r^^r^n^ 

ij- "-i^pecxea. Since som^e m.easures 

3uch as '£RRCJ are measures of errors on a test while 
others are measures of positive performance. However, a 
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result difficult to explain, is the sizable correlation 
of the two concept measures ERRCJ and FOCCJ. One of the 
is the number of errors an S makes in the process of 
identify the concept, while the other is an indicator 
of the 5's use of a focusing strate'gy. Yet, ^ this 
result appears more reasonable in the light of the 
assumption niade using the number of errors. It was 
that the number of errors was assumed to be an index 
•of the number of exemplars used by S to identify the 
concept. Since the focus score is calculated only 
on those exemplars the S uses before he solves the 
concept, a positive correlation between the measure of 
error and focus strategy is possible. 

The mean magnitude of the intercorrelations of the 
problen-solvinfr criteria with the SI tests was .12 with 
a range from -.25 to .30. Of the 234 correlation 
coefficients between these two batteries. 111 did not 
exceed the magnitude (.11) necessary for a significant 
difference from zero. Four measures - money earned, 
days used, and the information score from The New B ike, 
and the Verbal Haae total solutions - demonstrated a 
consistent pattern of low to moderate correlations with 
the SI tests. The New Bike inconsistency score, the 
total time and total trials from the Verbal Maze 
problem, and the total elapsed time for the concept 
problem presented consistently zero correlations with 
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the SI tests. Thus it would appear that the possibility 
of comon underlying factors is not high. In order 
to delineate the relationships among the variables, a 
nu:r.bar of analyses vere carried out on the correlation 
matrix. These analyses will be reported in the 
folicv;ing sections. 

i< ||g||li| ILanal^^ 

In order to explore the predictability of the problem- 
r^olving criteria given the SI tests, step-wise regression 
analyses were carried out for each one of the problem- 
solving criteria, using the SI tests as predictors. 
Multiple correlations and percents of variance accounted 
for were calculated for each criterion. These figures 
were based' on an optim-un predictor set for each cJiterion 
Which was determined by F tests on the change in multiple 
correlation.. Table 13 summarizes the results of the 
regression analyses. Two Particular points should be 
noted. First,, the, F test should be construed as a decision 
function rather than an exact test. Second, the multiple 
regression procedure takes advantage of all sources of 
variance including error variance. Thus multiple R's 
may be spuriously inflated. , 

,For the problem-solving instrument, The New Bike , 
the multiple correlations (R) ranged from a high of .43 
for the amount of money earned to .25 for the incon- 
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Table I3. Prediction ,of problem-solving criteria from 
SI ability measures. 



Criterion 



The New Bike 
Money earned 

Days used 



.43 
.37 



Information score .^5 

Strategy score .30 

Inconslsteney 

score .25 

Kumber of ateps .32 

The State Fair 

Number of steps ,21 

Verbal Mas^e «1 

Total time .18 

Total trials .16 

Total solutions .ii7 

Concept LURB 

Number of errors ' .33 

Focus score ;13 

Total elapsed time .23 



% predictable 
variance 



19 
13 
20 

9 

. 6 
10 



^3 
2 
23 

11 
2 

5 



SI predictors 
contributing 
slgnif leant 
variance 



NU, LI, RA, 
UTFLU, LR 

NU, LI, LR, 

UTPLU 

NU, RA, LI, 
LR 

NU, LI, RA 



UT 

NU, LJ, RA, WC 

JOA, LI 

AT, RA, WC 

CA, SO 

RR, UT, LR, 
PPT, LI 

LR, UT, UTPLU 
WC 

PPT, RA, WL, 
LR 
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sistency .core. The percent of variance accounted for 
- the criterion variabic.by the predictor set ranged 

from 19% for mpney earned to B% *or, • ' . 

jr iin.u to ^or the inconsistency 

score. I-cr every variable except .inconsistency 
=core, the SI test Ue^^J^ ,,3, 
perfor..anc=. The 31 tests Learnei^nformation and 
SSiatolAlternatiyes, appeared in secondary or tertiary 
positions Of importance in four out of the si. optimum 
predictor sets. The fluency measure fro;» the Utility 
Test contributed significant variance to the prediction 
of xoney earned and day's used. Logical.Rea5onins was a 
significant predictor of money earned, days uVed. and th. 
information score. Anally, yor£_Coji£letion was a ' 
Significant out minor predictor of number of steps and 
H2-ual_Thin£s_ was the only significant SI predictor 
of the inconsistency score. 

The number of steps measure for The State Fair 
had an S Of .21 With its octimum predictor set, which 
accounted for only Hi of the variance. The optimum 
set consisted of iudsin^ec^Adafitotio^ and Learned 
Inf ornation. ' ' ' 

The total tir.e and total trials measures from - 
iBlh-U^azeJl had multiple K's of only .18 "and .16 
respectively accounting for only 3Vand 2% of the 
variances. Howe.ver the multiple K for the total 
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solutions measure was a respectable .^7 which accounted 
for 23% of the criterion variance. The significant 
predictor set for this variable consisted of five SI ' 
teste including Rernejv^ere^^ ^dEl]^^lYJIh2:m.^ 
Lo;;iGal Reasoning , ?lanej;ii^Jv^^ and Learned 
- Information . 

For the concept identification task LURB, multiple 
R's ranged from a high of .33 and 11% of the variance 
to a low of .13 and 2% of the variance. Number of errors 
had the highest R, and was predicted by three tests 
^-'ith Logical Reasoning being the most , important . Since 
so little variance was predictable for the other two 
measures, the predictor sets were judged to be of minor 
importance . 

To summarize the regression analyses, no multiple R 
exceeded .50 and thus less than 25% of the criterion 
variance was predictable from the SI tests. However, it 
is important to recognize that this. may have been due 
to more than one cause, since the multiple R between a 
criterion and ^a predictor set is also dependent on the 
reliabilities of all measures, and the problem-solving 
criterion reliabilities were generally low. 

Structural analyses 

. As was stated in the Methodology chapter, a three- 
fold approach was taken to the question of underlying 



Er|c 118 



108 



Structure. The variable intercorrelations were analyzed 
by conventional factor analysis, canonical correlation 
analysis, and extension of the problem-solving criteria 
into a predetermined SI factor structure. 

Conventio nal factor analysis .- The first stx^uctural 
analysis carried out on the data was an iterated 
principal factor analysis extracting 31 principal 
factors. The eigenvalues and cumulative percent variance 
accounted for by each of the factors are giveii in Table 
14. The scree test (Cattell, 1965) was carried out by 
plotting the eigenvalue against' factor number for each 
of the factors extracted. This plot appears in Figure 11. 
This test proposes that eigenvalues of factors with a. 
higii proportion of error variance v;ill form a straight 
sloping line or "scree slope." From the plot it was 
judged that the "scree slope" began at "the sixth factor 
so the first five factors wore judged to be relatively 
error free. The matrix of correlations was then 
subjected once again to an iterated principal factor 
analysis which extracted five factors. These factors 
ware then rotated to a varimax criterion. The factor 
structure and communalities are reported in Table 15. 
This solution was judged not completely adequate in 
terms of simple structure since a number of tests and 
measures such as the Utility Test,' Controlled A*ssociates. 
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Figure 11. Plot of eigenvalues V3, factor nurJjer for 31 principal factors. 
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Expressional Fluency , and several measures from The 

New Bike had important loadings on more than one factor. 

This varincx solution was then subjected to an 
oblique rotation by the Prcinax procedure (hendrickson 
and /r'hite, 1956). Table 15 presents the primary 
factor loading matrix. Table 17 gives the factor 
intcrcorrelations and Table 18 reports the test- 
factor correlation matrix. 

Comparing Tables 15 and 16 it is evident that the 
Promax solution yielded a slightly simpler structure 
in terms of factor loadings. The loadings for the 
two. measures from the Utility Test increased slightly 
in factor IV and decreased in factor I. The major 
improvement was in the hyperplane count (loadings of 
magnitude less than .10) which improved for each 
factoz'. In addition, most tests had an important loading 
on only one factor. However, the factors yielded by 
Prornax procedure were no longer orthogonal. Table 17 
indicates that factor I is most correlated with factors 
II, IV and V, yet the magnitude is quite moderate (,34). 
Facto- II is slighuly negatively correlated with factor 
III and ■ essentially uncorrelate4^with factors IV and V. 
Factors IH, iv and V are eas-en'Tfially uncorrelated . It 
would appear then that the oblique rotation left most 
of the factors orthogonal K'hile moving factor I to a 

.1,2 
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Table 16. 


Primary factor matrix for the 


five 


factor 




Promax 


solution . 














Factor 






Measure 


I 


II 


III - 


IV 


V 


UTFLU 


.23 


-.05 


-.09 


. 59 


.06 


UTFLX 


.18 


.01 


-.02 


.56 


- . 02 


CA 


. .49 


-.OU 


.00 . 


. 37 


. 14 


EF 


. 39 


-.05 


.10 


.28 


.05 


RR 


. 7U 


-.05 


. 05 


.11 


- ; 0 6 


AT 


.6U 


-.05 


. 06 


.lU 


-.16 


UT 


.57 


-.02 


-.10 


.09 


. OU 


SO 


.51 


.05 


-.05 


.03 


-.01 


SA 


.37 


.05 


-.02 


. OU 


.lU 


JOA 


.44 


-.06 


-.02 


.03 


.16 


NU 


.57 


.12 


. 00 


.19 


-.66 


WL 


. 70 


.01 


. 09 


.07 


.00 


WC 


. 77 


-.05 


. 00 


.lU 


-.01 


PFT 


.60 


-.13 


-.OU 


.18 


.17 


LI 


. 32 


.17 


.01 7-::- 


-.15 


. OU 


LR 


.58 


.02 


-.11 


.03 


-. 08 


SP 


.59 


.01 


. ou 


.07 


-.16 


RA 


.6U 


.08 


.11 


. OU 


- . 09- 


VENB 


.09 


.87 


.00 


. 09 


.06 


DUNB 


. 06 


.83 


-. ou 


.10 • 


. OU 


INFNB 


.07 


.9U 


. 01 


. 01 


-.02 


STRNB 


-.07 


.93 


. 00 


.01 


-.06 


INCONB 


-.09 


.08 


-.02 


.lU 


.30 


NSTPNB 


-.lU 


.98 


. 02 


. 05 


.02 


NSTPSF 


-.01 


.02. 


. 08 


.08 


-.39 


TMSPY 


-.21 


.00 


-.07 


.01 


.38 


TTRSPY 


-.19 


.09 


.13 


.19 


.28 


TSSPY 


.43 


.09 


-.12 


.05 


.lU 


ERRCJ 


.OU 


.00 


.83 


.11 


-.38 


FOOCJ 


.03 


.00 


. 78 


.01 


. 06 


TMCJ 


.06 


.03 


. lU 


.15 


.11 
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slightly oblique position with respect to all of the other 
factors. 

In order to understand the psychological meaning of 
the five factors, it is necessary to examine the test- 
factor correlation matrix. Table 18, whick presents 
this matrix, reveals that while the primary factor 
structure became simpler through the oblique rotation, 
a greater degree of test overlap on factors is present. 
Factor I appears to be a general test perforniahfe factor. 
All measures with reliabilities greater than .30 have 
important correlations with this factor*. Factor II 
appears to be primarily a test specific factor having 
high correlations. with The New Bike measures, though the 
New Uses is also moderately correlated. This is 
probably due to correlated errors. Since these same 
tests also have important correlations with factor 
I, the correlation of the two factors is explained. 

Factor III is another test specific factbr which 
had high correlations with error and focus scores from 
the concept identification problem.^ The low negative 
correlations of this factor with factors I and II 
are most probably a result of the low negative correlations 
of a number of SI tests and The New Bike with this factor. 
This in turn is . most likely due .to the fact that the two 
measures assess degree of poor performance on the task. 
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Factor IV appears to represent a divergent production 
factor. Four of the seven tests which had important 
correlations with this factor have been identified by 
Guilford (1967) as divergent production tests. Two of 
the remaining tests. Apparatus Test and New Uses , call 
for "original" thinking by S in order to successfully 
perform on the tests. ''Original" thinking has often, 
been identified as divergent in nature. Finally, 
Word Completion has a moderate correlation with this 
factor, probably due to the semantic nature of all the,_. 
other tests. Factor IV has its highest correlation 
with factor I, most probably because a general test 
performance factor affects performance on these divergent 
production tests. 

Factor V has moderate correlations with several 
tests in both the SI tests and problem-solving criteria. 
This factor appears to indicate that the SI abilities of 
evaluation of semantic transformations ( Judging Object 
Adaptations ) and cognition of semantic systems ( Plane 
Flight Test ) are related to problem-solving measures of 
the errors made in problem solving ( The New Bike in- 
consistency score, number of steps from The State Fair , 
and number of errors in the concept problem). The 
measure of total time in the Verbal Maze problem 
also had an important correlation with this factor. > 
Factor V was moderately correlated with only factor I, 
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Table 17. Factor intercorrelations . Five factor Promax 
solution for the SI tests and problem- solving 
criteria. 



Factor 

Factor II ' m ly y 

I •34 -.21 .34 .31 

II -.21 .10 -.03 

III -.11 .08 

IV ' -.11 
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which can be accounted for by the important correlations 
°^ Judging Object Adaptations . Plane Flight Test , and 
number of errors in the concept problem with both factors. 

To summarize the results of this conventional 
factor analysis, it revealed little about underlying 
relationships between human abilities as specified by the 
SI tests and the problem-solving criteria. Only factors 
I and V showed any crossover between the two sets of 
'tests. However Factor I explained little since, it appeared 
to be a general test performance factor. Only factor V 
presented some evidence that the human abilities, 
evaluation of semantic transformations and cognition 
of semantic systems, are related to errors in problem- 
solving tasks. 

Canonical correlation analysis . Since the conventional 
factor analysis revealed little of the relationships 
between the set of human abilities tests and the problem- 
solving criteria, it was decided that a different, more 
suitable approach would be employed. This approach 
chosen was canonical correlation analysis which was 
designed by Hotelling (1936) to analyze relationships 
between sets of variables. 

Five canonical variates were extracted from the v 
set of SI tests and the problem-solving criteria. The 
pertinent information concerning these canonical 
variates is presented in Table 19. The first two 
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canonical correlations were significant according to the 
'Wilks lambda test. However, five canonical variates 
were extracted in order to account for a maximuin amount 
of variance in the two sets. The five canonical 
variates extracted 60% of the variance of the SI tests 
and 48% of the variance of the problem-solving criteria. 
The first two variates were the only ones to extract 
appreciable variance from the first set, while only 
the first canonical variate extracted appreciable variance 
from -the second set. 

The redundancy figures reported in Table 19 were 
the amounts of' variance extracted by the canonical 
variates from one set that were predictable from the 
corresponding canonical variates of the other set. 
Thus for the first pair of canonical variates which 
were correlated .66, the proportion of variance predictable 
in the second set from the first set is .074, while the 
proportion of variance in the first set predictable 
from the second set was .136. The total proportion of 
extracted variance predictable in the set of problem- 
solving criteria from the SI tests was .116. The reverse 
accounted for a total proportion of variance of .156. 
Finally, the first pair of canonical variates accounted 
for 64% of the redundancy in the SI tests and 87% of the 
redundancy in the second set. This indicates that the only 
canonical variates of importance were the first extracted. 
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pair of variates in the analysis. 

In order to analyze the contributions of the 
various tests due to the first pair of canonical 
variates, the correlations between the tests and the 
canonical factors were calculated. This canonical 
factor structure is given in Table 20. Since the first 
canonical factor accounted for most of the redundancy, 
this was the only one which was closely examined. All 
the SI tests had moderate correlations with this factor. 
The test exhibiting the highest correlation was New 
Uses with a coefficient of --75. For the problem- 
solving criteria, several of the measures also had 
reasonable size correlations "with the factor. These 
measures included four of the measures from The New Bike , 
the Verbal Maze total solutions, and concept problem 
total errors. The pattern of correlations across the 
two batteries appear quite similar to the general 
test performance factor found in the conventional factor 
analysis. 

Extension analysis > Since the canonical correlation 
analysis also revealed little about the relationships 
between the SI tests and the problem-solving criteria, 
a third approach was attempted. This analysis attempted 
to answer the question: assuming that the SI tests 
represent a structure of human abilities, how do the 
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Tabic 20.. Test correiatlonc vM.th the 5 Indeperdent canonical 
factors. 

Canonical factors 
^^-^s^>re I ^^xj TTj TV 7 

SI TcITti ' 



♦JT?LU 3 

JTr LX " • 32 

CA -.62 

p.n .,53 

AT -o^ 

SO ^.63 

JOA . 

- -,75 

v;l -.62 

v/C ..60 

PFT ^.r,- 

S? 



VENB 
STRNB 

XSTPSF 

TTR6:?Y 
TSSPY 

FOCCj 
TMCJ 
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,62 
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.10 
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•J 0 
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cf t'.o LI ter;tj;:. 'l-io firc^t attcrrot ut reprocTucing this 
;":r-.cture i^;; r^ror^cd 'InMo Thia table ^/ives 

the rocults cf r... Lt-^ratc:! T'rJn:^!;;^! factor solution, 
a .::'^rcc tort ' en tl'.o ci^'cnve.i.U'^-i*^. , '■"'.td a varirc^y rotitiou 
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A ::cconc apprOiVch wa:^ t^^nr crrrioa out. In this 
n.rcJ.'/c~i.E all :'".:iCLOr .; voro '•^^r^d ^^lic;. cxtrd")ctcd po?iti*/e 
vcricrc'-"^ fro::; ti.o, ir.t::rcorr*:latiori -"""itrix. Thic 13 
1 ^ t o r V ar i ^"i ■ : >: r o t a t c u c o 1 ut i o"; . L o o. i» o r t e d in Ta 1^ 2 2. 
7hr.7.'-i factorc accou*"J:od for lu'y> of the variance in tno 
Zl tcttG. It Ig evident that th2 IP SI factorr: suppos- 
ed 1 /■ r o r 0 0 C- r. t c d *r •* thi c t c t b a t t e v e r 6 r o t f u 3. 1 v 
rc: *o r c T ^ r. t: e d • I .o v a v e r , t a e a i* i c t c r c o a p p o ar to 
r ^ ^ ^ ^ a c n t 1 3 ^"'a c t or ^ s irii la i * to t h o a fa c t o r ?• g a c .x f ic d 
by t:\a SI r:odel. :>;inc of t:';a 1? factor'^ >.ad a siaj;le 
test vr;ich loaded hi^^her than ar.y other test> tnoufht 
tv;c ol tl:era ldd\r.r:'"^ v*ere r^. lev. i^: r.ar^nitude (factor 
Xll :nd factor XIII) taat th.e factor:: could only be 
tentatively identified a.^ re;:reeer.tin;: the particular 
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tests with the highest loadings. Four factors had 
essentially' equivalent high loading for several tests 
and thus were identified as representing more than one 
SI factor. Undo ubt ably, the structure could have been 
improved by oblique rotation, but in conformance with 
Guilford's criterion of orthogonality, this structure 
was accepted as the most representative structure of the 
SI model that could be derived from available data. 

Using the factor structure given in Table 22, 
extension loadings of the problem- solving criteria on 
the 13 factors were calculated according to the procedures 
of Hosier (1938). This had the effect of adding a row 
to the factor loading matrix for each of the problem- 
criteria. These added rows are presented in Table 23. 
Almost all of the extension loadings are less than .30, 
indicating that most of the problem-solving criteria 
were not related to the factor structure. There are, 
however, a few exceptions. The number of steps and 
money earned from The New Bike are moderately related 
to factor IV, which was associated with the SI factor -of 
memory for semantic systems. The information score from 
the same test appeared to be related to factor VII, which 

« 

was associated with convergent production of semantic 
systems. The New Bike strategy score and the concept 
problem total elapsed time were related to factor X, 
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whr.O. accecia-ed with co^^nition of .er.antic 3yster..3. 

Firally, the total olap^od tir.e for the co.ncept 
problen war. aloo related to factor XIII u-hich wa^ 
ton-ativoly ^d-n~i:;.eo r.r, -.o.rory for ser.r.r.tic relation?. 

To c--,rr.?r^ii:o th- oxtcr:alon loadin;; cralyoiG , little 
of t::e rela-:ionci-p:n letwoon the CI tests and the proLlem- 
Polvir.n; cr.i-eria were revealed. Only seven lo.idinc 
exceeded .30, and these '...--0 widely scattered across 
t}ic factors. Ihe outcop.e once again appears to be that 
rhcre vas ec sent daily no rclationchips between the SI 
te5:t3 and the problc-ri-r-olvin;; criteria. 

Su:ur,ary 

This concludes the results cha-oter. The findings 
of -r::i reliaoility study ;.avo been r-ported. The data 
concerning tne CI teste and the proDlem-solving 
criteria have been described and subjected to four 
different types o.^ .analyses . The results of the re- ' 
grcssion analyses and the three nultivariate correlational 
analyses were .reported. In the next chapter these results 
will be GisGussed and conclvsions win be drawn. 
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Z/te cu.*'PC30 ci' tiiic ^^tuiv wai: to evaluate 
'■ro . l^?''n--nolvin:-; criteria v^hic": i\o.d coen dci;irnar aad 
devolcpod to •.^cr'j acucu^it-,}!:^ srTp;.^ proL;Iom-r:>olvir..q 
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'^11':. Ln tor.TJ of ::or:to^.t ri.'^Tplf r' alternate forrr,:; 
ro.liabilitv . It is c-/ide.^;--: frc^v t'.o resu2.ts reC'Crted 
in t:ir: oreviouo c.v:i:v!:or t/.£it no^-c the problen-^colvin^ 
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si:ice both the porforr.ance and tJ)e relative ranking of 
Ss v;ill caan.ire . .Learning apparently did take place for' 
practically all of the problem-solvinn measure in the ' 
ter.t-retost conditi-n 3in-- rh-?. -^c-ortod means generally 
chcnped aero?;:: testine scGsionn. Tor the alternate 
forn'.3, learnin.o; v-as ovide-t primarily for the Verbal 
Maze probleir.s. I- differential learninjr did occur, it 
could account forthe lo:.- test-retest correlations of 
nc^t of the Treasures. 

Another factor th.at could affect the alternate forr.s 
reliability is tl^.e content that each inctrunent sampled. 
The bai.-ic a^sum.pticn behind alternate forp.3 reliability 
ncasurcrnent is th-t the f.o forr-? measure tlie same thin.s. 
Forrs should, thuc , only diff';;r in superficial character- 
istics. In the original developmental effort, an attempt 
wa:^ made to construct parallel forms for each of the types 
of instruments. for the si-.iulated problems essentially 
the same problem maps were used for Doth the single and 
m.ultiple solution, form.s. -'roblem ,f;oals were essentially 
the same; to find rcmethinc^ missin- by searchinn; a 
variety of place.r. , and to collect a given am.ount in a 
minimum time in order to obtain som.e thing. In the concept 
tasks the forms vjere exactly the sam.e . Only the target 
concept was changed in terms of the relevant dimensions. 
The Verbal Haze probler.s used the same problem maps for 
both forms. Only the names '.-.-ere changed from one .form 
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solver should utili-.e r.ore ir.f or.T.zt.ion in solvin- the 
problen. The nur;.::er of .teps also appears to be valid 
because the better prcblem-solver should take fewer 
■^t'o. .olvo OTcbl^r. ;iharo ::::,y ha some question 

^ibout the Gtratery cocre and inconsistency score, however 
The strategy score reflects the ccnfornance of the S's 
pattern of cnoices wi~h the sure strategy. But this 
sure strategy n^ay not correspond to the S's conception 
of a .bisst strategy, since the co.-jnon assumption is that 
t;.e best strate.ey involves the fewest steps. The sure 
=3tratesy .Taarantees that t.ie prcblcTr. v.-ill. be solved in 
ti.e nininium of days, rather than in the, fewest steps. 
Thus S3 rr.ay not be even atte;nptir..c: to follow a sure 
ct^-atery Knic;i cau-es t;-ie conf orr;-:ince measure to lose 
meaninc. Tne inconsistency score supposedly reflects 
the logical errors S ;iiake in follov.-inc directions in 
the problem. ilov-ver thir: reasure :nay reflect errors 
fro- two sources' rnisunderstandinc given directions and 
accidental rando.T errors due to such sources as turning 
tc the wrong page. V];is second source is not systernatic, 
and therefore the inconsistency score may reflect randoni 
as well as systematic vari-r^'.nce . 

Tor the concept proble.^.s, the number of errors and 
focusing score are recor^nized measures of concept 
identificrition often appearing in the research literature. 
Hovever, tne scanning score r.ay be less valid in this 
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typo of sit*;-:* ti::!-. * r**.ovg;i Ir a rcco.r^nizea (Bourne, 
et ri.l., 1"J72} ccncopt identification stratej^y. It io 
ci cc.^ple^: L^tr\,j tc;- y V''jq*.;irin,r; the 1' tc carry all possible 
-joncoptr^ r^r'n'^'^at-T^ .^^'or: focur. rx.amplc in his head^ 

r.ni c^ii-innt'- :r:^-c:ific re:! bilit ir as ■■o lcok3 at 
CoCA •jxorr:?i'^r . ;.t :^cc:-.:r prc>^ble tnat 3:. in r: nin/sle 
oorceot identif ic": tion tr.:^-, . v:;ian "triev cire not faniiliar 
wir'-: thr^ tyr^o of tr.3/, vcvlc not er.nloy such a co^nplex 
Gtrr.tc^r^y- '>]iu'j tro icarninr; !:;coro na;^ in r;?oGt casssj 
only reflect acc i d ,?n':r:il scruanccsf. cf exemplars ra.ther 
th'iin co*n:f:c icuc ^^tratef y . 

the Vcr.-ai -'azc.^. rornurcc aoper.r to posse";o 
faco Vc-lid.ity. ".''r. tul tir^; is a r^^.a jure of hovr efficient 
t'i-*i nroirler -c^o^.v::--^ ir:. Tctnl tri-.ls io a measure 
cf tre crror:^ t>'^:. 3 ""r,:-cec in ::olvip:; the -vrobleiT.. And 
total !^clut:-n5 r::fl'-:cr3 "*\o " ' ':7 a:jility to find a 
n jT*^^* of i * ''^o r 3 ' t i-'o ' L^tio^^ 

"'hcrofcro^, fror: tac standpoint of validity, it v:cv.ld 
.:.pp"-'Vr thrt tiio ^ir/jl.!: t':^d "^roblcm.-.- and the concept 
idcr.ti f ication ta':'''z coulc* je r.orr:ov*hat affected by the 
Vc l ic.ity d 'Dr i vn d 
th-^} crh'U'' hc^c^ do no 
con'^' Ldc^aticn . Tnia 



'.o '"erbal XazcG • on 



:o affected bv this 
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:ho ?,cv' rc 1 '.cbi li.t ier for the .s : "•'.v laired 
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From the -bovG d:.scu.23ion it is evident that a 
number of conditions existed which could adversely, 
affect the reliability, coeificients for the probleni- 
so-ving criteria. r^ince the reliabilities vore ouite 
-lev;, it is reasonable to e.ssuir.e tviat some of these 
factors were active in loverin^- the reliability of the 
different measures. Regardless of the causes of the 
c:ecr3T.ents in reliability, the important outcome of 
•chis study was that the problem-solving criteria exhibited 
consistently low reliabilities. There low reliabilities 
v^lli adversely affect all attempts to investigate the 
problem-solvinr: behavior sampled by these measures, as 
will be seen in tne fcllov/in- sections. 
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The Problem-Solving Abilities Study 
The primary outcome of this study was that few 
relationships were evident between this particular set 
of problem-solving criteria and the set of tests 
representing a number of Structure of Intellect (Guilford, 
196 7) factors. However, the relationships that were 
evident will be discussed in the following sections. 

Regression Analyses . 

The regression analyses were the most fruitful in 
terms of relationships revealed. These analyses were 
designed to determine the predictability of the problem- 
solving criteria from the SI test. Each of the problem- 
solving criteria was significant correlated with at 
least one of the SI tests. Multiple correlations were 
not great, the greatest being .47, and thus large amounts 
of variance in the criterion variables were left un- 
explained. Optimum predictor batteries ranged from 
one to five SI tests. 

' Three SI tests were significantly related to all 
of The New Bike measures except the inconsistency score. 
These were, in order of importance. New Uses , Learned 
Information , and Related Alternatives . New Uses represented 
the SI ability of convergent production of semantic 
transformations. Learned Information represented the 
SI factor of memory for semantic systems, and Related 
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Alternatives represented - -.e SI a-ility of menory for ' 
sen:-.ntic i.-nplicat^ons . i^rj^ appears to be logically 
related to perforn^.arce on this instrument in that the 



the- r.K'on inforrna-Zion in order to arrive at the best 
declGion, convergent prcducrion of semantic trans- 
fer. -nation 3 . -The lo-ical rolaticnsnip of Learned 
--Jl£2riI2l:±l?il ^^'^ ^^S}J:t^^^i^ to performance 
on this instrunient seems to lie in tne necessity for 
tiic C to recall -the structure and raeaninc; of information 
given in previous steps ir order -co make decisions in 
the present step. Ihis appears to be tne same abilities 
as described by memory for semantic systems (structure) 
and implications (meaning). 

In the concept identification task, only the number 
of errors nad an important amount of variance predicted 
by the optirium set. T];ree SI tests; i^osical^soning^^ 
^IL-Iii^iyJiilin and ytjJ.ity_Tc:st_- fluency, were 
significantly, and negatively related to number of 
errors. L ogical Heasoni n.f^ appears to be logicallv 
related, in that semantic raasonin- is necessary to 
interpret fecdbacK and determine -which dim.ensions are 
relevant. LIir^-:£2v:_^i2in£:ji does not appear to be 
loGioally related joecause of the 31 factor it represents, 

but rather throuro -{-ne f^rr t-'it^-^ -^-hn ~>.^+. j 

ij,K^^.^^.x ..it. racT ZAa ^ ^otn mstrumentG depend 
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on the S's ability to notice pictorial details. 
Utility Tes t - fluency may be related to performance on 
the concept identification task in that it is necessary 
in both tasks to generate a number of possible solutions 
to the problem-divergent production. 

A number of SI tests were also related to the total 
solutions measure of the Verbal Maze instrument. The 
significantlx^reiated, tests were Remembered Relations . 
Unlikely Things, Logical Reasoning . Plane Flight Test , 
and Learned Information. The underlying cause of relation-' 
ships appears to lie in the fact that the Verbal Maze 
is a semantic system, a relation among relations 
where the relation is "can talk to". Thus Remembered 
Relation _s (memory for semantic relations) and Learned 
Information (memory for semantic systems) may be related 
in that they measure the recall of semantic relations and 
systems. These correspond to remembering the name 
pairs and the connections among name pairs. Unlikely 
Things (evaluation of semantic systems) appears to be 
related in that the problem task requires the S to 
judge the adequacy of his solutions which are semantic 
systems evaluation of semantic systems. The two 
reasoning tests. Logical Reasoning (evaluation of semantic 
implications) and Plane Flight Test (cognition of semantic 
systems), appear to be logically related tothe task in 
that the task requires reasoning of the form: if A 
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can talk to B and B can talk to C then one can pass 
a message from A to C. This type of reasoning encompasses 
both semantic and general reasoning specified by the 
two tests. 

From the above discussion it is evident that a 
small number of relations existed between the SI tests 
and the problem-solving measures. These relations 
lend some weight to the assertion that some human 
abilities are important to problem solving. However, 
it is important to remember that the ability measures 
that were related to the problem-solving criteria 
predicted only a small portion of the criterion variance. 
Thus the relations are generally weak. 

Structural analyses . 

The factor analytic procedures employed in these 
analyses generally revealed very little about the 
relationships between the SI tests and the problem- 
solving criteria. The purpose of these analyses was 
to determine the underlying factors common to both the 
SI tests and the problem-solving criteria, and in this 
way to elucidate relationships between the two batteries. 

Conventional Factor Analysis . The conventional 
factor analysis revealed five factors, none of which 
resembled SI factors. The first factor was a general 
test performance factor, similar to a general intelligence 
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factor, which had loadings of all the SI tests and 
at least one measure from each of the problem-solving 
criteria. The second and third factors were instrument 
specific and represented the concept identification 
task and The New Bike w ith New Uses . The fourth 
factor represented divergent production of semantic 
products. Only the last factor demonstrated any 
crossover between the two batteries. However, this 
factor made little logical sense, and since the important 
factor- test correlations were barely greater than .30, 
the factor was judged unimportant. Thus the conventional 
factor analysis revealed nothing new concerning the 
relationships between the two batteries. 

Canonical correlation analysis . The canonical 
correlation analysis yielded only one important canonical 
factor. The test-factor correlations revealed that 
this factor was highly similar to the general test 
performance factor found in the previous analysis. 
The only new information this analysis added was that 
very little of the general performance variance in 
the problem solving criteria was predictable from the 
SI tests. 

Extension loadings analysis . The purpose of this 
analysis was to discover how the problem-solving tests 
fit into the SI factor structure supposedly represented 
by the SI tests. However, considerable difficulty 
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was encountered in reproducing the SI factor structure. 
This structure vas approximated only when the question- 
able procedure of extracting all of the variance by 
factors was employed. The extension loadings provided 
little new information. This analysis revealed only 
that Learned Information appeared to be related to The 
New Bike measures. 

Possible explanations for the poor results . The 
poor results of all these analyses can be traced back 
to the correlation matrix of the SI tests and problem- 
solving criteria. These correlations were generally 
quite low, and a large number did not exceed a 
magnitude which would be significantly different from 
zero. Thus the expectation of factors across the two 
batteries could not be high. In turn, these low 
correlations can be directly attributed to the low 
reliabilities of the problem-solving criteria. According 
to classical test theory (Lord and Novick, 1968), the 
correlation between two tests is limited by the square 
root of the product of the reliabilities of the two 
tests. Thus the low reliabilities of the problem- 
solving criteria severely limit the possible magnitude 
of correlations between them and the SI tests.; The 
possible explanations for these low reliabilities were 
discussed in the previous section. This appears to be 
the primary reason for the lack of evident relationships 
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in the multivariate correlational analyses. 

Several other more general factors may have also 
affected these analyses. In addition to those factors 
which could affect the reliability of the problem- 
solving criteria, characteristics of the specific 
tests and the sample may have affected the outcome. 

The New Bike placed great deal of emphasis on the 
reading ability of the Ss. It is possible that the ^ 
importance of reading ability overshadowed the effects 
of all other abilities. Thus the reading ability of the 
S may have dominated performance on the task, to the 
exclusion of most other abilities. However, the low 
correlations of The New Bike with Word Completion 
argue that this possibility is not highly probable. 

Another factor which may be characteristic of all 
of the problem-solving tasks is the complex nature of the 
problem-solving involved. The possibility exists, 
that the measures derived from the instruments did 
not suit the complexity of the tasks. If problem- 
solving is a complex interaction of a number of human 
abilities, it may not be reasonable to assume that 
• --gross measures which count events across the entire 

span of the problem-solving episode will reflect 
accurately the functioning of these abilities. All 
the measures derived from the problem-solving tasks 
sampled behavior across the entire task, so it. may be 



ERIC 



155 



that the importance of some abilities may have been 
masked by the gross nature of the measures. 

A factor which may have affected the concept 
identification tasks is the nature of the task- 
content. The S had to identify a two dimensional 
geometric concept. All exemplars were geometric 
. figures . Yet none of the SI abilities represented by 
the SI tests dealt with figural rather than semantic ^ 
content. If abilities focusing on figural content were 
of primary importance, few members of the SI test 
battery would be related to performance on the problem- 
solving task. 

A characteristic of the sample may have also 
affected the relationships between the two sets of tests. 
Guilford's SX tests were developed primarily using 
samples of servicemen, college students, and high 
school students. The question therefore arises: 
Do these tests measure the same abilities among fifth 
grade children as they do among adults? Some evidence 
exists (Torrance, 1966b; Wallach j t Kogan, 1965; 
Getzels g Jackson, 1961) that children possess divergent 
production abilities at a young age. However, it is 
open to question if such tests as Logical Reasoning , 
Apparatus Test, and Judging Object Adaptations elicit 
the same responses among adults as they do among children. 
It is quite' possible that the vocabulary component of 
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these tests is much more important for children than it 
is for adults. If this was true, then adult abilities 
measured by these tests would not be present in the 
scores of the Ss in the sample . Thus the relationships 
between the adult abilities and the problem-solving 
behaviors would not be present . 

One final contention concerning problem solving 
by Newell and Simon (1972) may have also influenced the 
lack of relationships. They conterid in their theory of 
problem solving in a human information processing system 
that: 

A few, and only a few, gross 
characteristics of the human IPS 
(information processing system) are 
invariant over task and problem 
solver (underlining added) 

This implies that there are not a large set of human 
abilities which are important to all problem solving 
situations, but rather that the abilities employed 
vary from task to task and person to person. If this 
contention wef^e true, then performance on the probi^em- 
solving criteria of this study would depend both on the 
task and the problem solver . Since all analyses were 
aimed at extracting common characteristics across in- 
dividuals, this dependence on the individual would 
effectively remove the possibility of the analyses 
yielding such general characteristics. Thus, few, if 
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any, relationships would be found between the SI tests 
and problems-solving criteria. 

It would appear from the above discussion that 
there are several possible explanations for the lack 
of significant relationships between the SI tests and 
the problem solving criteria. One or several of them 
may have, in fact, been true for this study. However, 
concrete evidence exists for only one explanation. 
From the first study it was known that the reliabilities 
of the problem-solving criteria were quite low, and 
thus the correlations with the SI tests were limited. 
The resulting low to zero correlations effectively 
prevented the elucidation of many possible relationships. 

Inability to reproduce a Structure of Intellect 
factor structure . One final question must be dealt 
with. Why was an SI factor structure not reproducable 
from the intercor^r^^l^'tions of the SI tests? One very 
simple explanation is that in order to produce a given 
factor it is necessary to have at least two tests which 
represent that factor, and they must have a higher 
correlation between them than each has with all other 
'%ests. This study had only one test representing each 
' of the chosen SI factors, and thus the likelihood of 
producing SI factors was extremely small. 

Even if the proper number of tests were present^ 
there is some question of whether or not the SI structure 
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could be reproduced using strictly empirical methods.- 
To quote Horn (1970) 

The fact that subjective methods were 
allowed in research upon which the 
SI model is based invokes the uneasy 
feeling that perhaps - in what degree 
one can only surmise - what are 
referred to as distinct abilities are, 
in reality, only drawingboard vectors 
.created by overf actoring and rotated 
to fill otherwise unfillable cells in 
the SI cube ... 

When this type of objection is combined vjith the 
possibility that some SI tests may not measure the same 
things in children as they do in adults, the possibility 
of reproducing SI factors using test performances of 
fifth graders seems even more remote. 

Summary and Conclusions 

The project encompassed by this thesis developed 
a set of problem-solving tasks which were believed to 
be more in line with a set of characteristics for the 
ideal problem- solving task. However, the reliability 
study revealed that these tasks failed one important 
criterion - they were not reliable measures of the 
behaviors they were sampling. In spite of this, these 
problem-solving tasks were administered to a large 
group of children along with 17 human ability measures as 
specified by Guilford* s (1967) Structure of ^Intellect 
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model. The analysis of these results revealed few 
relationships between the SI tests and the problem- 
solving criteria. The few relationships that existed 
indicated that memory abilities were important to 
two of the three tasks, a measure of convergent 
production of semantic transformations was related to 
performance on the simulated problems, and reasoning 
and evaluation abilities were related to performance 
on the concept identification and Verbal Maze tasks ^. 
A primary stumbling block in investigating possible 
relationships vas the low reliability of the criteria • 
though some other factors may have been operative. 

The primary conclusion that must be 'drawn from this 
project is that the unreliability of the problem-solving 
criteria severely limited the study and essentially 
prevented the study from fulfilling many of its purposes. 
The question of what human abilities are important to 
problem solving thus remains open. The basic design 
of this study is still capable of providing information 
regarding this question, but a number of improvements 
need to be made. First, the reliability of the problem- 
solving criteria need to be raised to acceptable levels. 
Second, an even wider sample of problem- solving behaviors 
need to be included as marker tests. Third, the population 
of interest should be adult problem-solvers instead of 
children. With these improvements, it is quite possible 
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that important contributions could still be made to our 
knowledge concerning human problem solving. 
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ir:pLiCATiC':>is roK ELI;CATI0N 

Tne goaa. of Phai>G III or th^a proj^ict was to construct 
an empirically La-^-e.. rr.oael fi)V instruction riunian problem 
solving. This model v/as to luil.I ou tnr- results of Phase II 
combined with ^uiuelix\e^ ana priiiciples derived from tlie research 
literature witn rerpect tc u:'cblf.ru solving., li^at is^, ^hase II 
was to have est-ai^lislied witn -sor^r: rtrli ai^xiitv , v.r.icti of the 17 
specific ol aL/ilities were relzitaa to several different aspects 
of problem solving. Inese aspects were reflected in the different 
measures derived from the problem-solving criteria. It was 
assumed tnat t::ese reiat lonsrdps between the specific abilities 
anc tiie problera-solvinp, measure:, would serve a:, preliminary 
evidence that enhancing, tne anpropriate specific abilities 
would lead to more efficient and accurate problem solving. 
Tne model for instruction in problem solving v;ould tucn have 
involved a set of empirically based principles for the nurturance 
of t'nose importan-^. human abilities. Since it was evident that 
even if tiie second phase were successful the m.odel could not be 
complete based solely on tnis project ^s results, relevant principles 
derivable from tbe prol lein-solvin^-;' literature were also to be 
included. 

However, as is obvious from t)\e precedin? chapters. Phase II 
was essentially unsuccessful in establisliinf. a structure of 
specific SI abilities wiiich were related to the various aspects 
of problem solving reflected in the measures derived from. the 
problem-sol vinp criteria. Tnus, the dilemma of constructing 
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an Inadequate r::0'':-:l or r,f raining from mocel consrruction 
wao preijentod. L i nco the evidence in t.ie proclem-solving 
literature ccncerning t:;e i/nportdnce of diiferent S'pecific human 
abilities V as f rci^':irj^.-ntar:^ -aid lacoirr. iete , a model based solely 
on tiie cxict:!:-' ll:.erature was judged to be inadec.;uate for 
instructi'onai ^^urposeci. T.iic lack of defi.nite empirical evidence 
fron: tne researcn 1 ixerature , tof;etiapr v;ita th*;i lack of results 
from P/idse II of tiie px^oject, led to th^? decision tnat no 
useful model could be ':onstructed at this time. That xs, 
no useful and empirically leased mode.l for instruction in specific 
cognitive aptitude^ could be constructed frorr. evidence presently 
available . 
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